Getting Paid
How to track your time and get paid on time.
How Payment Works on Project Lizard
Fellows are paid by the number of approved annotations they submit regardless of time spend per annotation. To be paid, an annotation must clear three checks, in order:
- Review layer: A score of 2 or below sends it back for a rewrite, unpaid until it passes.
- Quality audit: Approved annotations are spot-checked. If an audit finds it doesn't meet guidelines, it's returned for revisions.
- Client check: Once a batch of tasks is delivered, the client will run a difficulty check on each annotation. If the model answers a question correctly >40% of the time, it is not accepted. Only accepted annotations are paid.
Because of this three-step process, expect a delay between submitting and getting paid.
Client audit / Audit_II returns: These tasks currently route back to Audit_I and look like normal Audit_I tasks. Always check the Audit_II feedback tab. If the feedback is incorrect, select Yes for Do you want to rebuttal Audit_II feedback? at the bottom of the Audit_I feedback for the specific prompt. Explain the issues clearly and thoroughly, then approve and submit as a passing Audit_I task. If the feedback is correct, fix it yourself and resubmit without sending it back to the annotator. If you are stuck with a truly unusable or unsalvageable image, report it in the reviewer/auditor Slack thread. Scrapped tasks are unpaid because they are not tied to a submitted annotation: do not log a Handshake AI task, and abandon any already started. Do not spend too much time on scrapped work. See the Reviewer Playbook for the full workflow.
Tracking Time
You'll use two platforms:
- Handshake AI: where you claim Handshake AI tasks, track your time, and submit your work
- SuperAnnotate: where you claim tasks and complete your annotations or reviews
The submission steps below are for annotators. The new Model Stump Check (Stump the Model) in the Handshake AI Shadow Task is currently for annotators only and is not part of review/QC or audit workflows. Reviewers and auditors do not run it, verify a passing result, or apply its three-wrong-answer threshold; follow the Reviewer Playbook for review logging. If this changes, there will be an announcement and the playbook will be updated.
Claim a task
Open your Project Details page in Handshake AI and claim a task.
Review the task instructions
After you claim the task, review the instructions carefully before you begin.
Work within SuperAnnotate
Complete the task directly in SuperAnnotate. Track your time in Handshake AI as you go.
Submit the task
Once an annotation is complete, include its Task ID, correct event_group_uuid, uploaded SuperAnnotate image, copied prompt, copied rewrite answer, and matching Short Answer or MCQ tag in its separate Handshake AI task. The image, prompt, and rewrite must match SuperAnnotate exactly. Click Continue and pass the Model Stump Check before submitting that Handshake AI task. The model must answer incorrectly 3 or more times; if it answers incorrectly fewer than 3 times, revise the prompt, rerun the model and Check My Work, copy the updated information, and check again. Do not retry an unchanged prompt. See the full requirements.
Confirm your time
Confirm your tracked time, edit if it's inaccurate, and click Submit Task to log your work. Repeat for each annotation. Submit the full SuperAnnotate task only after all individual annotations are complete and all their Handshake AI tasks are submitted.
Each annotation needs its own separate Handshake AI task. Never log more than one annotation in a single task. For annotators, run it alongside the annotation you're working on in SuperAnnotate, and include all required inputs. Submit it once you're done making changes to that annotation and it has passed the Model Stump Check. A failed check makes the annotation invalid: do not submit it; it will not be paid. Submit the full SuperAnnotate task only after all individual annotations are complete and all their Handshake AI tasks are submitted. Passing the check does not guarantee payment; review approval is still required. The new check does not apply to review/QC or audit workflows. If you skip the Handshake AI task, Handshake AI can't record the work, and you may not be paid for any annotation you didn't log. Skipping the Handshake AI task is not a way around throttles either: unlogged work isn't recorded, so it can't be credited or paid.