Two engineers review a large technical drawing on a worktable in an engineering office.

Research Lifecycle

A research lifecycle connects an industrial question to the evidence needed for a decision. It starts with a defined problem, moves through experiments appropriate to the uncertainty, and reaches a pilot or controlled release only when justified. Findings can send the work back to an earlier stage.

This page describes a proposed workflow for Aperture research. It does not establish that every stage has occurred. Each project record should state its dated status, methods, setup, findings, limitations, and next step.

Define The Problem and The Hypothesis

Describe the physical setting

Specify the equipment or process, affected users, operating conditions, and decision that needs support. Identify what is inside the study boundary and what remains under an existing operating procedure. Record constraints such as access windows, communication limits, data permissions, and who can authorize changes.

Make a testable claim

A hypothesis states what change is expected and how it will be evaluated. For example, a proposed pump study could ask whether a condition estimate helps identify a defined fault earlier than an existing threshold method. Define what counts as the fault and how its onset will be established independently. Do not choose a favorable success measure after inspecting the results.

Establish The Baseline and Evidence Plan

Select a relevant comparison such as the current procedure, a fixed rule, or an established model. Give both methods comparable inputs and operating conditions. Agree evaluation measures before trials, including failures and operator intervention, as well as the intended improvement.

Prepare data and ground truth

Record where observations came from, their units, timestamps, quality limits, and permitted use. Ground truth is the independent reference used to assess correctness. A prediction should not be judged solely against labels generated by the same model. Where the reference is imperfect, describe its uncertainty and how disagreements will be reviewed.

Separate development and evaluation data by time, machine, or site where appropriate. Retain configurations and versions so another researcher can reconstruct the comparison. Missing coverage may require collecting more data or narrowing the hypothesis before testing proceeds.

Use Digital Tests To Challenge Assumptions

Choose recorded-data replay, synthetic cases, simulation, or a combination suited to the question. Exercise ordinary operation and defined degraded conditions. Keep calibration data separate from validation data where feasible, and identify the operating range in which the model is credible.

If a method succeeds only because it receives information unavailable in the physical system, revise the experiment. If performance changes under plausible model variations, record that sensitivity and use it to prioritize physical measurements. Digital tests support preparation; they do not establish physical performance by themselves.

Evaluate In A Defined Physical Testbed

A testbed study uses specified equipment, instrumentation, supervision, and operating limits to examine the real response. Confirm permissions and stop conditions before introducing changes. Record the baseline and candidate behavior with an independent reference, including timing, losses of observation, and interventions.

In the pump illustration, a test might compare a diagnostic estimate with independently established condition evidence under permitted operating states. This is a proposed example, not a completed Aperture experiment. A disagreement could require checking the instruments, revising the model, or returning to the hypothesis.

Scope A Supervised Pilot

A pilot examines a bounded use in an agreed operational context. Define duration, equipment, users, responsibilities, access, and evaluation criteria. Establish who can pause the trial, how the existing workflow is restored, and what evidence is required before any broader use.

Evaluate operator usefulness and economic value alongside technical behavior. Include review time, integration effort, false alerts, and disruptions where relevant. A pilot can show that a technically effective method is unsuitable for the workflow, or that more evidence is needed before a value claim is justified.

Review The Evidence and Choose The Next Step

Compare findings with the original question

Report the baseline, sample scope, repeated runs, uncertainty, and failures. Separate physical measurements from simulated results. Explain whether the hypothesis was supported under the tested conditions and which alternative explanations remain. An inconclusive result is a reason to revise the study, not to imply success.

Return loops when evidence is insufficient

Unreliable labels return the work to ground truth. Poor transfer returns it to model assumptions and data coverage. Excessive intervention returns it to workflow design. A changed question requires a revised evaluation plan. Preserve earlier versions so the reason for each change remains traceable.

Release Only For A Defined Use

A controlled release, where justified, needs an approved configuration, responsible owner, monitoring plan, change process, and rollback route. Document the supported operating conditions and unresolved limits. Later drift, equipment changes, or failures can reopen evaluation rather than automatically trigger an unreviewed update.

NIST AI RMF provides background on risk management across AI development and use. It does not replace project-specific evidence or confer certification. Project status should distinguish proposed, in progress, demonstrated in a defined testbed, evaluated in a supervised pilot, and deployed for a defined use.

Plan An Evaluation

Share a nonconfidential description of the physical problem, current baseline, available observations, access constraints, and the decision the research should inform.

Scroll to Top