Documentation

For the full implementation bounty, we require documentation of the task family and how you tested it. From our perspective, the main difficulty in assessing tasks is quality assurance. It’s helpful if you can provide evidence that you’ve anticipated and mitigated potential issues with your task.

The task template has more details about how to fill out each of these files.

• Task family documentation
  • Fill out meta/eval_info.json with basic information about each task in the family.
  • In meta/summary.md, write one or two paragraphs about what the agent is supposed to do and how it will be scored.
  • In meta/detail.md, write a more in-depth explanation of the task family.
• QA documentation
  • Fill out meta/qa/qa.json with basic information about the QA run.
  • Add the participant’s progress log in meta/qa/progress.md.
  • Add the participant’s review document in meta/qa/review.md.
  • Other high-level information can go in meta/qa/README.md.
  • Add any other relevant files (for example, code written by the participant) in the meta/qa folder.