Testing Strategy

merql uses an oracle-driven testing model. Each test scenario defines a known set of inputs (three database states), a deterministic expected result (the oracle), and compares the actual merge output against that expectation. 32 scenarios cover clean merges, conflicts, identity resolution, data types, scale, and edge cases.

# Run the full verification suite.
./bin/verify-all

The commands on this page run from a checkout of the merql repository after composer install. The development scripts in bin/, the scenarios/ corpus, and the tests are not part of the Composer package. The development dependencies include PHPUnit 13, which needs PHP 8.4 or later, even though the library itself runs on PHP 8.2.

Oracle-driven testing

There is no existing reference tool that performs three-way database merge, so the oracle is computed from known inputs rather than generated by an external tool. Each scenario has a deterministic expected result that can be verified by logic:

1. Setup -- create three database states (base, ours, theirs) with known content.
2. Oracle -- compute the expected merge result from the known inputs.
3. Actual -- run merql's merge on the three states.
4. Compare -- the actual result must match the expected result exactly.

Known inputs (base, ours, theirs)
         |                    |
    Deterministic          merql merge
    computation
         |                    |
    Expected result       Actual result
         |                    |
         -----> Compare <-----
                  |
            Pass or Fail

Running tests

Full verification

The verify-all script is the final gate. It runs static analysis, coding standards, PHPUnit tests, and the oracle regression suite:

./bin/verify-all

This must pass before any work is considered complete.

Oracle regression suite

Run all 32 scenarios:

./bin/test-regression

Run a single category:

./bin/test-regression --category clean
./bin/test-regression --category conflict
./bin/test-regression --category identity

Categories are the directory names under scenarios/: clean, conflict, identity, types, scale, and edge. An unknown category prints No scenarios found and exits successfully. --category can be repeated to run several categories.

The script also accepts --jobs, but it ignores the value and runs scenarios one after another (#5).

Run with minimal output (pass/fail only):

./bin/test-regression --fast

Single scenario

Test one scenario through the full pipeline:

./bin/test-scenario column-level-clean

Individual oracle steps

./bin/oracle column-level-clean    # Compute expected result
./bin/actual column-level-clean    # Run merql, capture output
./bin/compare column-level-clean   # Diff oracle vs actual

Unit tests

composer test:unit         # Isolated component tests
composer test:integration  # Integration tests (SQLite, plus MySQL when configured)
composer test              # All PHPUnit suites
composer test:oracle       # Oracle regression corpus (bin/test-regression)
composer verify            # analyse + cs + test + test:oracle

composer test runs PHPUnit only. The oracle PHPUnit suite points at tests/Oracle, which holds the scenario runner classes but no test cases, so the 32 scenarios run only through composer test:oracle, composer verify, or ./bin/verify-all.

The MySQL integration tests are skipped unless MERQL_MYSQL_HOST, MERQL_MYSQL_DATABASE, and MERQL_MYSQL_USER (plus optionally MERQL_MYSQL_PASSWORD and MERQL_MYSQL_PORT) point at a test database.

Code quality

composer cs            # PSR-12 coding standards check
composer cs:fix        # Auto-fix coding standards
composer analyse       # PHPStan level 10 static analysis

Scenario categories

clean (8 scenarios)

Clean merges with no conflicts. Both sides may change data, but never the same column of the same row.

conflict (6 scenarios)

Scenarios that must produce conflicts.

identity (4 scenarios)

Row identity edge cases.

types (6 scenarios)

Data type handling across all column types.

scale (4 scenarios)

Performance under load.

edge (4 scenarios)

Edge cases and boundary conditions.

Scenario structure

Each scenario is a directory under scenarios/<category>/<name>/ with a scenario.json configuration:

{
    "name": "column-level-clean",
    "category": "clean",
    "description": "Both sides modify same row but different columns",
    "tables": ["test_posts"],
    "expectations": {
        "conflicts": 0,
        "operations": 3,
        "changeset_match": "exact",
        "merge_match": "exact",
        "sql_match": "semantic"
    }
}

For conflict scenarios, the expected conflicts are specified in detail:

{
    "expectations": {
        "conflicts": 1,
        "conflict_details": [
            {
                "table": "test_posts",
                "primary_key": {"id": 42},
                "column": "title",
                "ours_value": "Welcome",
                "theirs_value": "Greetings"
            }
        ]
    }
}

The ScenarioRunner pipeline

ScenarioRunner::run() orchestrates a single scenario:

1. OracleCapture::compute() builds three snapshots from the scenario's JSON data and runs the merge.
2. ScenarioComparator::compare() checks the merge result against the expected values in scenario.json.
3. Returns pass/fail with a list of failure messages.

use Merql\Tests\Oracle\ScenarioRunner;

$result = ScenarioRunner::run($scenario);
// ['name' => 'column-level-clean', 'pass' => true, 'failures' => []]

ScenarioRunner::runAll() runs every scenario in sequence:

$results = ScenarioRunner::runAll($scenarios);

Adding a new scenario

1. Create a directory: scenarios/<category>/<name>/.
2. Create scenario.json with the scenario metadata and expectations.
3. Define the three database states in the JSON configuration: base data, ours mutations, theirs mutations.
4. Run the scenario: ./bin/test-scenario <name>.
5. Verify the full suite: ./bin/verify-all.

Code quality standards

merql enforces:

• PHPStan level 10 -- strict static analysis with no baseline exceptions.
• PSR-12 -- coding standards checked by phpcs.
• Oracle regression -- all 32 scenarios must pass.
• Unit tests -- isolated tests for each component.

The verify-all script runs all four checks in sequence. No partial sign-off is accepted.

Continuous integration

CI runs coding standards, PHPStan, composer test, the integration suite against MySQL 8.4 through mysqli, merge primitive coverage, and composer audit on every push and pull request to main. It does not run the oracle regression corpus (#6), so run composer test:oracle or ./bin/verify-all locally before relying on a change.