Research & Innovation
Research is how we find out whether an idea deserves to be built.
At HRIMH, research is an applied discipline. Its purpose is to reduce uncertainty before significant engineering investment is made — to establish, with evidence, whether an approach works for a specific problem, under realistic conditions, at an acceptable cost and level of risk.
This page describes the method used to do that. It is presented as a working methodology rather than a record of results: HRIMH does not cite publications, patents or experimental findings here, and will publish such material only when it exists and has been reviewed.
Methodology
A seven-step protocol from question to decision
The protocol is designed so that every step produces a written artefact that can be reviewed by someone who was not involved. That property — reviewability — is what distinguishes structured research from informal experimentation.
Problem formulation
Define the user, the context, the constraints and the desired outcome precisely enough that success can be recognised.
A well-formulated problem names a specific group of people, the situation in which they encounter the problem, the constraints they operate under — time, skill, connectivity, regulation, budget — and the observable change that would count as improvement. Vague formulations such as “improve efficiency” are refined until they describe a measurable outcome for an identifiable user.
- ? Who exactly is affected?
- ? What would measurably change?
- ? What constraints are fixed?
Artefact
Problem statement and success criteria
Prior-art and technology assessment
Establish what already exists before building something new.
Existing products, open-source tools, published methods and established practices are reviewed against the formulated problem. Often a mature, simpler approach already solves most of it; sometimes the review reveals why previous attempts failed. Candidate technologies are compared on fitness for this problem — not on novelty or general popularity.
- ? What has been tried?
- ? Why did it succeed or fail?
- ? Is a simpler approach sufficient?
Artefact
Landscape review and candidate shortlist
Feasibility analysis
Assess data, infrastructure, dependencies, cost and risk.
Feasibility asks whether the necessary data exists and can lawfully be used; whether infrastructure can support the approach at the required scale; what external services or vendors the system would depend on; what it would cost to build and to operate; and which technical, operational, privacy and ethical risks it introduces. Some ideas end here — which is a successful use of the process.
- ? Is the data available and usable?
- ? What will it cost to run?
- ? What could go wrong?
Artefact
Feasibility assessment and risk register
Experiment design
Define hypotheses, evaluation criteria and meaningful baselines before testing.
Each experiment states a falsifiable hypothesis, the metric by which it will be judged, the threshold that would count as success, and a baseline for comparison — often the current manual process or a simple heuristic. Defining these in advance prevents the common failure of choosing metrics after seeing the results.
- ? What do we expect to observe?
- ? Compared with what?
- ? What result would change our decision?
Artefact
Experiment protocol
Prototype development
Test the critical technical assumption with the smallest possible artefact.
A prototype exists to answer a question, not to resemble a finished product. It targets the assumption that, if false, would make the whole idea unviable — whether a model can extract the required fields, whether users will complete a flow, whether an integration can meet latency requirements. Everything not needed to answer that question is deliberately omitted.
- ? What is the riskiest assumption?
- ? What is the least we can build to test it?
Artefact
Focused prototype
Validation
Assess accuracy, reliability, usability, security and operational suitability as applicable.
The prototype is evaluated against the protocol. Depending on the system, this may include accuracy on a held-out dataset, behaviour under realistic load and failure conditions, task completion with representative users, a review of security exposure, and an assessment of whether the system fits the operating context. Results are reported in full, including where the approach underperformed.
- ? Did it meet the defined threshold?
- ? Where did it fail?
- ? For whom does it not work?
Artefact
Evaluation report with limitations
Engineering transition
Document limitations and decide whether the concept merits further development.
The final step is a decision, made explicitly: proceed to production engineering, revise and re-test, or stop. If proceeding, the gap between prototype and production — security hardening, error handling, monitoring, accessibility, documentation, support — is identified and estimated. Known limitations are carried forward into product design, not lost.
- ? Proceed, revise or stop?
- ? What is the production gap?
- ? Which limitations must users know about?
Artefact
Decision record and transition plan
Evaluation framework
Evaluating AI components honestly
A single accuracy figure conceals more than it reveals. A classifier that is ninety per cent accurate overall may be far less accurate on the rare, high-severity cases that matter most. A generative system may produce fluent answers that are not supported by its sources.
HRIMH evaluates AI components across several dimensions, chosen according to the task. Results are reported per dimension, alongside the conditions under which they were obtained, so that decision-makers can judge fitness for a specific purpose.
| Dimension | Question answered | Typical method |
|---|---|---|
| Task accuracy | How often is the output correct on representative inputs? | Labelled held-out set; per-class precision and recall rather than a single aggregate figure. |
| Calibration | Does reported confidence correspond to actual correctness? | Compare confidence bands with observed accuracy; set review thresholds accordingly. |
| Robustness | How does performance change with noisy, unusual or adversarial inputs? | Perturbed and edge-case test sets; inputs in other languages or formats. |
| Groundedness | Are generated statements supported by the provided sources? | Source attribution checks; sampled human review of claims against references. |
| Subgroup performance | Does the system perform worse for particular users or categories? | Disaggregated results by relevant segment; investigate any significant gaps. |
| Stability over time | Does behaviour change as data or model versions change? | Pinned versions, regression suites and re-evaluation on every model update. |
Readiness
A convincing demonstration is not a production-ready system
Demonstrations are valuable: they show that an idea is possible. But the distance between a demonstration and a dependable system is routinely underestimated, and that gap is where most of the cost, risk and engineering effort actually lies. Naming the differences explicitly makes the gap visible early.
- Inputs
- Demonstration: Curated examples that work well
- Production: Whatever real users submit, including malformed data
- Failure
- Demonstration: Avoided or edited out
- Production: Expected, detected, logged and recovered from
- Users
- Demonstration: The builders, who know its limits
- Production: People with no knowledge of how it works
- Load
- Demonstration: One request at a time
- Production: Concurrent use, peaks and sustained volume
- Security
- Demonstration: Usually out of scope
- Production: Authentication, authorisation, abuse prevention
- Data
- Demonstration: Sample or synthetic
- Production: Personal, sensitive, subject to retention rules
- Change
- Demonstration: Rebuilt freely
- Production: Versioned, migrated, backwards compatible
- Accountability
- Demonstration: None required
- Production: Audit trail and identified human owner
System reliability
Reliability is the property users notice most when it is absent. It depends on handling partial failure — a slow dependency, a dropped connection, a malformed input — without corrupting data or leaving users uncertain about what happened. Research prototypes are examined for the failure paths a production system would need, so the cost of reliability is understood before the decision to proceed.
Human oversight
Oversight is designed, not assumed. Effective review requires that reviewers see the evidence behind an output, have time and authority to disagree with it, and are not presented with so many items that review becomes a formality. Experiments measure not only model accuracy but whether the combined human-and-system process produces better decisions.
Privacy-conscious design
Experiments are designed to use the minimum personal data necessary, with synthetic or de-identified data preferred where it does not compromise validity. Where real data is required, access is restricted, retention is time-limited, and data sent to external model providers is assessed against applicable obligations before any experiment begins.
Have a technical question that needs a careful answer?
HRIMH is open to research collaboration and feasibility work where a well-designed experiment could inform a significant decision. Describe the question, the context and any data or constraints involved.