ci-cdembeddedautomation

Integrating RocqStat into CI for Real-Time Systems: Pipeline Examples and Best Practices

UUnknown

2026-03-10

9 min read

Automate WCET checks with RocqStat in CI: fail builds on timing regressions, export metrics to Grafana, and adopt baseline-driven gating strategies.

Stop losing time to unpredictable timing: integrate RocqStat into CI for real-time reliability

Teams building safety-critical and real-time systems in 2026 face two linked problems: fragmented timing data across test runs, and no reliable way to gate CI on timing regressions. This drives costly debug cycles and risky releases. This guide shows how to automate WCET (worst-case execution time) analysis with RocqStat directly in CI pipelines, break builds when timing drifts, and surface timing trends in dashboards so teams can act early.

Why this matters now (2026 context)

In January 2026, Vector Informatik announced the acquisition of StatInf's RocqStat technology to unify timing analysis with code testing. This signals a wider industry shift: testing and timing analysis are being consolidated into CI/CD toolchains, not left to manual labs. Modern embedded CI is increasingly cloud-native, AI-assisted for WCET estimation, and expects traceable, reproducible timing metrics as first-class artifacts.

"Timing safety is becoming a critical ..." — Vector statement on integrating RocqStat into VectorCAST (Automotive World, Jan 2026).

What you'll get from the examples below

Concrete pipeline examples: GitHub Actions, GitLab CI, Jenkins.
Practical steps to fail builds on timing regressions.
Patterns to export RocqStat metrics to Prometheus/Grafana dashboards.
Best practices for reproducible timing measurements in embedded CI.

CI design principles for automated WCET and timing regression control

1. Treat timing like functional tests

Make timing metrics first-class CI artifacts. Store baseline WCET and benchmark results alongside binary artifacts. Use deterministic test selection and pin CPU affinity to reduce noise.

2. Baseline, compare, then gate

Every PR or nightly build should compare measured WCET against a stored baseline. Decide policy upfront: a strict gate (fail build) for >X microsecond regression or a staged warning that requires triage.

3. Use both measurement and static analysis

RocqStat supports measurement-driven analytics and static WCET estimation. Use static analysis to reason about theoretical upper bounds and measurement runs to validate and catch environment-specific regressions.

4. Control environment and quantify noise

Run multiple iterations, warm up caches, isolate CPUs, disable turbo and frequency scaling on test nodes, and capture standard deviation. Use statistical thresholds (e.g., p95, p99) instead of single-run numbers.

Concrete pipeline patterns

Pattern A — GitHub Actions: fail PR on timing regression

This example assumes RocqStat provides a CLI (rocqstat) and a compare subcommand that returns non-zero on regressions. The pipeline does: checkout, build, run instrumented test, analyze, compare with baseline artifact.

# .github/workflows/timing.yml
name: Timing CI
on: [push, pull_request]

jobs:
  timing:
    runs-on: ubuntu-22.04
    steps:
      - uses: actions/checkout@v4

      - name: Setup toolchain
        run: |
          sudo apt-get update && sudo apt-get install -y build-essential
          # setup cross toolchain if required

      - name: Build firmware
        run: make all

      - name: Run timing measurement
        run: |
          # produce a RocqStat-compatible traces/artifact
          ./tests/run_timing.sh --output=rocq-trace.json

      - name: Install RocqStat
        run: |
          # example install; replace with vendor-specific setup
          curl -L https://downloads.example.com/rocqstat/cli/latest -o /usr/local/bin/rocqstat
          chmod +x /usr/local/bin/rocqstat

      - name: Analyze with RocqStat
        run: |
          rocqstat analyze rocq-trace.json --output=rocq-results.json

      - name: Download baseline
        uses: actions/download-artifact@v4
        with:
          name: rocq-baseline

      - name: Compare vs baseline
        id: compare
        run: |
          # will exit non-zero if regression > threshold
          rocqstat compare rocq-results.json rocq-baseline/rocq-results.json --fail-threshold=200us || echo "REGRESSION"

      - name: Upload results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: rocq-results
          path: rocq-results.json

Key notes: use artifacts to persist baselines, and use the RocqStat compare exit code to gate PRs. Add an annotation step to post timing summary to the PR for reviewers.

Pattern B — GitLab CI: nightly regressions and trend exports

Nightly runs are ideal to capture stable baselines and export metrics to a time-series DB for visualization.

# .gitlab-ci.yml
timing-nightly:
  stage: nightly
  tags: [timing-node]
  script:
    - make all
    - ./tests/run_timing.sh --output=rocq-trace.json
    - rocqstat analyze rocq-trace.json --output=rocq-results.json
    - rocqstat export rocq-results.json --format=prometheus --endpoint=http://pushgateway:9091/metrics/job/rocqstat
  artifacts:
    paths:
      - rocq-results.json
  only:
    - schedules

Exporting directly in Prometheus format to a Pushgateway or via an exporter lets Grafana panels show p95, p99, and maximum execution times over time.

Pattern C — Jenkins declarative pipeline: hardware-in-the-loop (HIL) runs

Use Jenkins when you need to orchestrate HIL rigs. The pattern below shows running tests on a lab device, pulling trace files back, analyzing, and updating a baseline repository if approved.

pipeline {
  agent any
  stages {
    stage('Checkout') { steps { checkout scm } }
    stage('Build') { steps { sh 'make all' } }
    stage('Deploy to DUT') { steps { sh './deploy_to_board.sh' } }
    stage('Run HIL timing') { steps { sh './hil/run_timing_on_dut.sh --out /tmp/rocq-trace.json' } }
    stage('Analyze') {
      steps {
        sh 'rocqstat analyze /tmp/rocq-trace.json --output=/tmp/rocq-results.json'
      }
    }
    stage('Compare') {
      steps {
        script {
          def status = sh(script: 'rocqstat compare /tmp/rocq-results.json baselines/rocq-results.json --fail-threshold=0.5ms || echo REG', returnStdout:true).trim()
          if (status.contains('REG')) {
            error 'Timing regression detected — failing build.'
          }
        }
      }
    }
  }
}

How to store and manage baselines

A robust baseline strategy reduces false positives and supports traceability:

Per-branch baselines: Keep baselines per stable branch and update them via gated change requests.
Per-target baselines: Maintain separate baselines for different hardware targets and kernel configurations.
Versioned artifacts: Store rocq-results.json artifacts in your artifact store (Nexus, Artifactory, S3) with signed hashes for traceability.

Dashboarding: visualize trends, not just single failures

Failing a build is useful for rapid protection, but trends show systemic drift. Export RocqStat metrics to a time-series DB and build these dashboard panels:

Max WCET over time (daily/nightly jobs)
p95 and p99 latency trends
Regression alerts (increase > X% over baseline in last N runs)
Per-test heatmaps to spot which test cases cause the most variance

Prometheus + Grafana example

Pattern: rocqstat export --format=prometheus -> Pushgateway or custom exporter -> Prometheus scrapes -> Grafana alerts. Example alert rule:

groups:
- name: timing.rules
  rules:
  - alert: TimingRegression
    expr: increase(rocqstat_wcet_max[7d]) > 0.10 * avg_over_time(rocqstat_wcet_max[14d])
    for: 1h
    labels:
      severity: critical
    annotations:
      summary: "WCET increased >10% in the last 7 days"

Show p99 with a Grafana query like: max_over_time(rocqstat_wcet_p99[24h]). Combine with a panel that lists recent PRs where the regression originated using links from CI artifacts.

Advanced strategies and caveats

Single-run variance will cause false failures. Use repeated runs and statistical comparisons (e.g., Mann–Whitney U test, bootstrapped confidence intervals) or supply rocqstat with multiple traces to produce stable estimates.

2. Reproducibility on shared runners

Cloud runners can introduce noise. Use dedicated runners with tuned RT kernels or container images with CPU isolation. Example kernel settings: set the CPU governor to performance, disable hyperthreading, and pin tests to isolated CPUs (isolcpus, nohz_full).

3. Combine static WCET with measurement for safety certification

For safety-critical systems, regulatory evidence often requires static WCET proofs. Use RocqStat static estimation to compute upper bounds and annotate measured results to demonstrate margin coverage.

4. AI-assisted regression triage (2026 trend)

With the rise of AI in developer workflows, 2025–2026 tools increasingly surface likely root causes for timing regressions (e.g., added I/O, changed scheduling). Integrate AI triage to prioritize PRs that are likely responsible for regression.

5. Security and data governance

Timing traces may include proprietary control-flow data. Secure artifacts with encryption at rest, sign baselines, and ensure CI nodes are hardened. When sending metrics to shared SaaS dashboards, redact sensitive identifiers.

Example: Annotate PRs with timing report and remediation hints

Create a small CI step that posts a compact timing summary to the pull request, with links to full RocqStat artifacts and suggested remediation items (e.g., identify changed files that correlated with increased WCET).

# pseudocode step
rocqstat summarize rocq-results.json --format=markdown > summary.md
gh pr comment $PR_ID --body-file summary.md

Make the summary actionable: show delta, affected functions, and recommended next steps (e.g., rollback commit, micro-optimizations, move to RT thread).

Operational playbook: from fail-fast to continuous observability

Start small: add timing checks to critical modules only.
Run nightly full-system timing suites to build baselines.
Gate PRs for critical modules with strict thresholds; enable soft warnings for others.
Automate triage: link regression to commits using CI artifact correlation.
Visualize trends and set alerts for signal drift, not only hard thresholds.
Review baselines regularly and require approved baseline updates via documented process.

Real-world example: automotive toolchain consolidation (2026)

Vector Informatik's acquisition of RocqStat (Jan 2026) signals consolidation of timing analysis into broader verification toolchains like VectorCAST. Expect smoother integrations where WCET analytics live alongside unit, integration, and model-based tests. For teams, this means less fragile glue code between tools and easier enforcement of timing contracts in CI.

Checklist: quick-start for embedding RocqStat into your CI

Install and validate RocqStat CLI on a dedicated timing runner.
Create an instrumented test that emits RocqStat-compatible traces.
Define baseline artifacts and store them in your artifact repository.
Add compare step to PR pipeline; define fail thresholds and notification policies.
Export metrics to Prometheus/InfluxDB for Grafana visualization.
Automate PR annotations and triage to accelerate fixes.

Final recommendations and pitfalls to avoid

Avoid single-run gating — use aggregated statistics and thresholds.
Beware comparing across hardware — maintain per-target baselines.
Keep CI nodes stable and documented — kernel options, CPU affinity, and thermal states matter.
Integrate timing checks early in the dev cycle; late detection costs far more.

Conclusion — build timing safety into CI, not after it

In 2026, timing analysis is no longer an offline lab exercise. Integrating RocqStat into CI pipelines gives teams the power to detect regressions early, defend releases with reproducible evidence, and visualize long-term trends that reveal real system drift. Use the pipeline patterns here to get started—fail fast where needed, but measure and observe continuously.

Actionable next step: Fork our sample pipeline repo (link in the CI job artifacts) and add a RocqStat compare step to one critical CI job this week. If you run into variability, follow the checklist above: isolate CPUs, warm caches, and run multiple iterations until the signal is stable.

Call to action

Ready to automate WCET analysis in CI and stop timing regressions from shipping? Try integrating RocqStat into one PR gate this sprint, or request a demo of the RocqStat-VectorCAST workflow to see how timing analytics integrate with your existing verification pipeline.

Unknown

Contributor

Senior editor and content strategist. Writing about technology, design, and the future of digital media. Follow along for deep dives into the industry's moving parts.