Position: Senior Software Engineer – Python (LLM Evaluation & Repository Validation)Type: Contractor AssignmentLocation: RemoteCommitment: 40 hours/week with some overlap with PSTEngagement Length: 3 monthsStart Date: Immediate – within 1 weekRole ResponsibilitiesDesign and develop verifiable software engineering tasks using public repository dataAnalyze and triage GitHub issues across widely-used open-source repositoriesSet up, configure, and manage development environments including DockerizationEvaluate unit test coverage, code quality, and repository robustnessRun, modify, and test real-world codebases to assess LLM performanceIdentify grounding issues, incorrect outputs, and weak reasoning in model evaluationsCollaborate with research teams to identify challenging datasets for LLM trainingContribute to expanding dataset diversity across languages and difficulty levelsLead or support junior engineers in repository evaluation and task creationRequirementsStrong proficiency in Python is mandatoryHands‑on experience with Git and DockerMinimum 3 years of software engineering experienceAbility to understand and navigate complex, large-scale codebasesExperience working with high-quality public repositories (5000+ stars preferred)Strong analytical thinking and problem‑solving skillsFamiliarity with software testing, debugging, and pipeline setupAbility to work independently in a remote environmentReliable system setup with stable internet connection#J-18808-Ljbffr