Testing and evidence
We test the platform in layers because a passing simulation test does not prove that account permissions, browser controls or a database deployment work. This is the procedure we can demonstrate; it is not a claim that every live user journey has already been tested. The separate API Testing and UI Testing pages describe the more detailed test cases.
Automated checks
The application repo has a Gitea Actions workflow at .gitea/workflows/quality.yml for pushes and pull requests. It checks JavaScript syntax, validates the JSON presets, checks preset determinism assumptions, runs Python backend/client/UI-contract tests, enforces at least 60% measured Python application line coverage, and runs four Godot headless test scripts. The Python coverage figure does not cover JavaScript or GDScript. A registered Gitea runner and access to the pinned Godot download are required for the CI jobs to finish.
Run the key checks locally from the application repository root:
node --check web/app.js
python tests/validate_presets.py
python tests/test_determinism_ci.py
python -m coverage run -m unittest discover -s backend/tests -v
python -m coverage report --fail-under=60
godot --headless --path . --script res://tests/test_determinism.gd
godot --headless --path . --script res://tests/test_experiment_features.gd
godot --headless --path . --script res://tests/test_stimuli_recording.gd
godot --headless --path . --script res://tests/test_live_doors.gd
The coverage commands need the Python coverage package; CI pins version 7.10.7. On 14 September 2026, local validation of the integration branch found six valid presets with no validator warnings. The previous local backend run passed 31 tests. These local checks are evidence for that branch only, not proof of a green Gitea run or a deployed Supabase configuration. Keep the actual CI link or screenshot with the sprint submission.
Manual checks before a release
| Area | A useful end-to-end check | Record as evidence |
|---|---|---|
| Account | Create an account, confirm email if required, sign in, reset password and sign out. | Build, browser, date, outcome and error if any. |
| Project access | Create a project from each available preset; add an existing account as lead, scoped researcher and observer; retry forbidden edits. | Account role, permitted module and server/database response. Do not publish private email addresses. |
| Template builder | On Ahmed's branch, draw region, wall and door; preview JSON; save; reopen the experiment in 3D. Test local and hosted paths separately. | Branch/commit, saved ID, validation or permission result. |
| Setup | Change seed, treatment, stimulus, environment suggestion and starting door; apply and save; reopen to check persistence. | Before/after values and saved version. |
| Run | Start, pause, step, reset, switch speed and camera; open/close a live door; confirm the intervention appears at the intended step. | Seed, step, event and replay reference. |
| Review | Download JSON/CSV, save a replay, play, seek and exit; verify the downloaded files open. | File schema/version and result, without personal data. |
| Layout | Try a wide screen, narrow screen, 200% browser zoom, keyboard navigation and a long error message. | Screenshot and whether any control is obscured. |
The live Supabase migration and Row Level Security rules need separate tests. A browser showing a hidden button is not evidence that a forbidden database write fails. Database functions, project membership, project scope and experiment scope must all be checked with real test accounts.
Feedback and bug handling
The team uses its Sprint board for work tracking and Gitea branches and pull requests for implementation history. A bug report should state the branch or deployed build, steps to reproduce, expected and actual result, screenshot or log, severity, owner and a test for the fix. Mark it closed only after retesting.
For stakeholder or user feedback, record the date, participant role, task they tried, what they said or struggled with, resulting decision, issue/commit and retest outcome. Summarize comments in the team's own words and omit private account details. The documents currently available do not provide a verified Sprint 2 interview or usability-test record, so we should not invent one to meet the rubric. The team must add the real evidence when those sessions happen.
AI Attribution: This test guide was drafted with OpenAI Codex assistance from the current workflow and test files. Local observations and any future user-feedback evidence need team verification.