Skip to main content

Automated Testing

Why automated testing is used​

The Virtual Rodent system contains deterministic simulation behaviour, experiment configuration, scheduled stimuli, treatments, recording, replay, persistence and backend services.

Changes to one part of the system can affect behaviour elsewhere.

Automated testing provides a repeatable way to verify important functionality whenever the project changes.

The automated test suite currently contains tests written in:

  • GDScript for Godot simulation and experiment behaviour; and
  • Python for backend API and environment-service behaviour.

UI-specific automated tests can be added to this testing section by the team members responsible for the user interface.

Testing approach​

Tests focus on externally observable behaviour rather than internal implementation details where possible.

The project tests areas such as:

  • deterministic simulation runs;
  • stimulus scheduling;
  • treatment events;
  • intervention recording;
  • experiment validation;
  • experiment migration;
  • save and load behaviour;
  • recording;
  • replay;
  • JSON and CSV export;
  • API authentication;
  • API authorization;
  • API security;
  • environment preset generation; and
  • external service integration boundaries.

The objective is not simply to confirm that code executes without errors.

Tests should verify that the produced result matches the behaviour expected by the application.

Godot automated tests​

Godot tests verify simulation and experiment functionality independently from the visual interface.

The current automated test files include tests for:

test_determinism.gd
test_experiment_features.gd
test_stimuli_recording.gd

These tests cover different parts of the simulation and experiment lifecycle.

Determinism testing​

Reproducibility is an important requirement for a behavioural simulation.

A simulation using the same experiment configuration and seed should produce the same result when executed again under the same conditions.

The determinism test runs equivalent simulations and compares their results.

The test verifies behaviour including:

  • final simulation state;
  • generated event sequence; and
  • recorded step data.

The test also verifies that the expected experiment events are generated.

A change that unintentionally changes deterministic behaviour should therefore cause the test to fail.

Experiment feature testing​

Experiment feature tests verify behaviour surrounding experiment configuration and persistence.

The suite includes testing for areas such as:

  • malformed experiment configurations;
  • validation errors;
  • schema migration;
  • malformed JSON;
  • simulation speed behaviour;
  • experiment save and load;
  • version changes;
  • reproducibility after loading;
  • no-op saves; and
  • replay round trips.

These tests help ensure that experiments remain valid and reproducible when they are stored and loaded again.

Stimulus testing​

Stimulus tests verify scheduled changes within an experiment.

The simulation preset contains stimuli such as:

  • light;
  • odour; and
  • sound.

Protocol entries activate or deactivate stimuli at configured simulation steps.

Automated tests advance the simulation and check that scheduled changes are reflected in simulation state and recorded events.

This verifies that stimulus behaviour depends on simulation steps rather than wall-clock time.

Treatment testing​

Treatment behaviour is also tested automatically.

The current experiment model supports:

  • medicine;
  • placebo; and
  • no-treatment conditions.

Automated tests verify that treatment administration generates the required event and that treatment information is recorded.

This ensures that treatment events form part of the reproducible experiment history.

Intervention testing​

Researcher interventions can occur while a simulation is running.

Automated tests verify that interventions are recorded together with the simulation step at which they become effective.

Recording interventions is important because a replay or exported result should contain actions that may have affected the experiment.

Recording testing​

The recorder stores simulation data for each recorded step.

Recorded information can include:

  • simulation step;
  • simulation time;
  • rodent position;
  • region;
  • active stimuli;
  • treatment events; and
  • interventions.

Automated tests verify that these values are captured as the simulation advances.

Export testing​

Simulation results can be exported for later analysis.

Automated tests verify supported output formats including:

  • JSON; and
  • CSV.

The tests inspect the generated output rather than only confirming that a file was created.

This helps verify that meaningful experiment information is present in the exported result.

Replay testing​

Replay tests verify that recorded runs can be saved and reconstructed without changing the experiment result.

Replay testing supports the requirement that completed simulations can be inspected again after the original run.

The automated suite tests replay behaviour at both the Godot experiment level and the backend API level.

Python automated tests​

Python tests verify backend services independently from the Godot simulation tests.

The current Python test files include:

test_app.py
test_environment_service.py

The backend tests use Python's automated testing tools and temporary test data.

API automated tests​

The API test suite starts a temporary backend server and performs real HTTP requests against it.

The tests cover:

  • password hashing;
  • authentication;
  • cross-paradigm authorization;
  • CSRF protection;
  • revision conflicts;
  • paradigm customization restrictions;
  • simulator command authorization;
  • command whitelisting;
  • replay creation;
  • environment preset authentication;
  • environment preset responses; and
  • environment error handling.

Detailed API testing documentation is provided separately in the API Testing page.

Environment service testing​

Environment preset generation is tested separately from the API endpoint.

This allows environment rules to be verified without requiring an HTTP request or live third-party service.

The tests provide controlled environmental conditions and inspect the generated experiment settings.

Daylight testing​

A daylight environment contains values such as:

  • temperature;
  • cloud cover;
  • precipitation;
  • wind speed;
  • weather code;
  • daylight state; and
  • solar radiation.

The automated test verifies that daylight produces an active light preset with a valid lux value.

It also verifies that the generated lux does not exceed the configured limit.

Night testing​

Night conditions are tested separately.

When the environment indicates night and no solar radiation is present, the generated preset should disable the light stimulus and set its lux value to zero.

This confirms that daytime and nighttime data produce different experiment presets.

Sound preset testing​

Environmental sound generation is tested using multiple controlled conditions.

The service tests include examples such as:

  • normal ambient conditions;
  • precipitation; and
  • stronger wind.

This verifies that the environment service converts broad weather conditions into suitable sound presets.

External dependency testing​

Normal automated tests should not depend directly on live third-party services.

The environment integration uses controlled responses during automated tests.

This provides the following boundary:

Automated Test
      |
      v
Virtual Rodent Environment Service
      |
      v
Controlled / Fake Weather Response

Live Application
      |
      v
Virtual Rodent Environment Service
      |
      v
Open-Meteo

This separation ensures that a test result represents the behaviour of Virtual Rodent rather than the current availability of an external provider.

A live integration test can be performed separately to confirm that the real Open-Meteo connection is working.

Automated testing procedure​

The standard procedure for running automated tests is:

  1. switch to the branch or commit being tested;
  2. install or activate the required project environment;
  3. run the relevant Godot tests;
  4. run the backend Python tests;
  5. review any failures;
  6. correct the implementation or test where required;
  7. rerun the failed test;
  8. rerun related tests to check for regressions; and
  9. confirm that the relevant suite passes before merging.

Running Godot tests​

Godot simulation tests can be executed in headless mode.

Examples:

godot4 --headless --path . --script tests/test_stimuli_recording.gd
godot4 --headless --path . --script tests/test_determinism.gd
godot4 --headless --path . --script tests/test_experiment_features.gd

Headless execution allows simulation tests to run without opening the Godot editor.

This is useful for repeatable local testing and future continuous integration.

Running Python tests​

The Python virtual environment should first be activated.

Example:

source .venv/bin/activate

The Python tests can then be run using the project's configured test command.

For example:

pytest

Coverage can also be collected using:

pytest --cov

When tests must be run​

Relevant automated tests should be run:

  • after implementing a feature;
  • after fixing a defect;
  • after changing shared simulation code;
  • after changing experiment configuration;
  • after changing persistence behaviour;
  • after changing backend endpoints;
  • before merging a feature branch; and
  • before a sprint demonstration or release.

Developers do not need to run every unrelated test after every minor edit.

However, changes to shared components should be followed by the broader relevant test suite.

Handling failures​

When an automated test fails, the developer should determine whether the failure is caused by:

  • an implementation defect;
  • an incorrect test;
  • an intentional behaviour change; or
  • a problem in the test environment.

The failing test should not simply be removed or disabled.

The expected process is:

Test Failure
     |
     v
Reproduce Failure
     |
     v
Identify Cause
     |
     v
Correct Implementation / Valid Test
     |
     v
Run Failed Test Again
     |
     v
Run Related Regression Tests
     |
     v
Pass

Regression testing​

When a software defect is fixed, an automated regression test should be added where practical if the defect was not previously covered.

The regression test should reproduce the behaviour that failed before the fix.

This prevents the same defect from being accidentally reintroduced later.

Test isolation​

Tests should not modify normal project data.

Backend tests use temporary databases and local servers.

External service tests use controlled data.

Simulation tests use known experiment configurations and deterministic seeds.

This allows tests to be run repeatedly without depending on state left behind by a previous run.

Future additions​

The testing section can be expanded as other team members complete additional project areas.

Possible additions include:

  • automated UI tests;
  • additional researcher workflow tests;
  • performance testing;
  • additional portal integration tests; and
  • further regression tests created from user feedback.

The existing testing structure is intended to allow these tests to be added without changing the overall testing approach.

AI Attribution: This document was drafted and edited with AI assistance, including ChatGPT and OpenAI Codex. The team reviewed it against the implemented project.