About the Company
SME Careers connects subject-matter experts, students, and professionals with flexible, remote AI training work such as annotation, evaluation, fact-checking, and content review. We partner with leading AI teams, and all contributions are paid weekly once approved, ensuring consistent and reliable compensation.
About the Role
We're hiring a Code Reviewer with deep Java expertise to review evaluations completed by data annotators assessing AI-generated Java code responses. Your role is to ensure that annotators follow strict quality guidelines related to instruction-following, factual correctness, and code functionality.
Responsibilities
- Review and audit annotator evaluations of AI-generated Java code.
- Assess if the Java code follows the prompt instructions, is functionally correct, and secure.
- Validate code snippets using proof-of-work methodology.
- Identify inaccuracies in annotator ratings or explanations.
- Provide constructive feedback to maintain high annotation standards.
- Work within Project Atlas guidelines for evaluation integrity and consistency.
Required Qualifications
- 5–7+ years of experience in Java development, QA, or code review.
- Strong knowledge of Java syntax, debugging, edge cases, and testing.
- Comfortable using code execution environments and testing tools.
- Excellent written communication and documentation skills.
- Experience working with structured QA or annotation workflows.
- English proficiency at B2, C1, C2, or Native level.
Preferred Qualifications
- Experience in AI training, LLM evaluation, or model alignment.
- Familiarity with annotation platforms.
- Exposure to RLHF (Reinforcement Learning from Human Feedback) pipelines.
Compensation
:
- $18 Hourly
Please make sure to add the JOB ID: Java when applying for the position.
Why Join Us?
Join a high-impact team working at the intersection of AI and software development. Your Java expertise will directly influence the accuracy, safety, and clarity of AI-generated code. This role offers remote flexibility, milestone-based delivery, and competitive compensation.