Bounty

We’re offering a cash bounty for high-quality task implementations. We pay at least $300 per hour we estimate it would take a professional to complete the task. We’ve paid out over $100,000 in bounties so far!

In addition to the bounty, anyone who submits a task implementation or a specification that we use can get an acknowledgement in the README for our task suite repository.

We’re also excited about this bounty as a way to find promising hires. If we like your submission, we may fast-track you through our application process.

Submit your finished task using the implementation form.

Guidelines for bounty amount

Difficult tasks take more effort to develop but also tell us more about an agent’s capabilities. Therefore, the bounty amount is based on estimated time for a human professional to complete the task family. Time estimates are made at METR’s discretion based on results from the submitted QA run and/or our own QA run.

As a baseline, we pay $300 per task-hour. If you submit a good task that would take 10 hours for a human professional, the bounty would be around $3,000.

Note the following caveats:

• Computation time doesn’t count. If the task family involves e.g. waiting 1 hour for a model to train, that doesn’t count towards our time estimate. If your QA run involves significant computation time, please report it separately.
• Repeated steps may count less. If the task family involves repeated or very similar steps (either within one task or across tasks), we’ll pay the full rate for the first repetition of each step. We may offer a reduced rate for the rest, if the repetition is valuable for minimizing noise.
• Steps missing QA may count less. We’ll pay the full rate for parts of the task which have evidence that they work correctly (from either a full QA run or one of the alternatives). We may offer a reduced rate for the rest, depending on our level of confidence that they work correctly.

For tasks that are especially good (unique, threat-relevant, easy to run) we may offer a bonus up to 50% of the base payment.

Payment logistics

We have gone to great lengths to make sure we can pay bounty-winners in ways that work for them, but occasionally it’s illegal or extremely impractical for us to pay a bounty, ie if your bank account is in a country under US sanctions (e.g. Russia) or if you’re in the US on a visa that doesn’t permit paid work. In these kinds of cases, we reserve the right to not pay the bounty. If you’re worried about this, please reach out to us at [email protected] before you start working on a task.