SOFTWARE TESTING

Quality isn’t a phase. It’s part of the architecture.

Quality engineering is a core NeuralSharks discipline. We design how a product is tested the way we design how it’s built: around its risks, its architecture, and the way it ships.

Testing at the end finds problems when they cost the most to fix. We put checks at every layer, from unit tests on business rules to load tests on the busiest journey, and wire them into the pipeline. Every change then arrives with evidence that it works.

01 / WHY TEAMS CALL US

The problems we’re usually hired to solve.

Most teams already test. The trouble is that the testing doesn’t give them confidence, speed, or either.

01

Customers find the bugs first.

Regressions reach production and are reported by users or support. Every release feels like a gamble, and hotfixes crowd out planned work.

02

Releases wait for a test cycle.

Days of clicking through a spreadsheet of test cases before each release. It’s slow, it misses things, and it grows with every feature.

03

The test suite cries wolf.

Flaky tests fail at random, so people rerun them until they pass. Real failures get ignored along with the noise.

04

Feedback takes hours.

The pipeline is so slow that engineers batch their changes and learn about breakages long after they’ve moved on to something else.

05

Nobody knows what’s covered.

Coverage is a percentage on a dashboard. Nobody can say which journeys are protected, or which risks were accepted on purpose.

06

Fine in testing, slow in production.

Performance, scale, and security are checked once before launch, if at all. The first real traffic peak becomes the load test.

02 / CI PIPELINE

Quality gates on every change.

Each stage runs the cheapest checks that can catch a fault at that level, and each has a gate. Choose a run to see what a gate stops, and select any stage to see what it checks.

feat(checkout): add discount codes

A feature passes every gate

A pull request adds discount codes to checkout. Every stage passes, so it reaches customers with the evidence attached to the pull request.

IN PROGRESSRunning Commit: stage 1 of 7.

01Running

Commit

  • Format and lint staged files
  • Type-check the changed modules
  • Pull request opened for review

GATEHooks pass and a second engineer reviews the change

ci › commitGIT HOOKS · ESLINT
$ git push origin feat/discount-codes
lint-staged · prettier, eslint
✓ 14 files formatted and linted
✓ pull request opened, review requested

An illustrative pipeline with representative timings and results. We size the stages to your product: a young product may need only a few of these gates, and that is part of the strategy.

03 / WHAT WE TEST

Systems with hard edges to test.

The kinds of products we test most often, and the part of each that usually needs the most care.

  1. 01

    SaaS platforms

    TenancyRolesBilling

    Multi-tenant products with roles, subscriptions, admin tools, and integrations.

    THE HARD PARTProving one tenant can never see another’s data, in every screen, export, search result, and API.

  2. 02

    Web applications

    BrowsersAccessibilityPerformance

    Customer-facing sites and applications used across browsers, devices, and connection speeds.

    THE HARD PARTBehavior that changes with the browser engine, the viewport, and the network, not with the code.

  3. 03

    Mobile apps

    DevicesOfflineUpgrades

    iOS and Android apps, native or cross-platform, together with the APIs behind them.

    THE HARD PARTDevice and OS fragmentation, interrupted sessions, poor connectivity, and users who never update.

  4. 04

    APIs and integrations

    ContractsWebhooksRetries

    Public, partner, and internal APIs, webhooks, and connections to third-party systems.

    THE HARD PARTConsumers you can’t see, and partner sandboxes that behave differently from production.

  5. 05

    Payments and billing

    MoneyProrationRefunds

    Checkout, subscriptions, invoices, refunds, and the reconciliation behind them.

    THE HARD PARTRounding, currencies, proration, and partial refunds, where nearly right is still wrong.

  6. 06

    Event-driven systems

    EventsOrderingIdempotency

    Queues, workers, real-time updates, and the processes that react to events.

    THE HARD PARTDuplicate, delayed, and out-of-order messages, which usually appear only under load or failure.

  7. 07

    AI features

    Evaluation setsGuardrailsRegression

    Assistants, search, classification, and generation built into a product.

    THE HARD PARTAnswers that vary from run to run, so pass and fail need evaluation sets rather than exact matches.

  8. 08

    Legacy systems

    CharacterizationSafety netRefactoring

    Products with years of behavior, few tests, and changes coming.

    THE HARD PARTCapturing what the system actually does today, before anyone changes it.

04 / TESTING DISCIPLINES

Thirteen kinds of testing, one strategy.

Each kind of testing answers a different question, at a different cost. A strategy decides which questions your product needs answered, and how often.

01 / MANUAL QA · THE QUESTION IT ANSWERS

Does it work, and make sense, for a person using it?

People still find what automation can’t: confusing flows, wrong copy, visual defects, and problems nobody thought to write a test for. We run exploratory sessions against written charters, test new features by hand while they are still changing, and write bug reports a developer can act on without a follow-up conversation.

  • Exploratory sessions with charters, time boxes, and notes
  • Manual checks for features too new or volatile to automate
  • Bug reports with steps, environment, evidence, and severity
  • Settled manual checks moved into automation

YOU RECEIVEExploratory session notes, manual test cases, and actionable bug reports

qa/bugs/BUG-0142.mdEXAMPLE ARTIFACT
# BUG-0142  Discount removed when delivery changes
Severity: High    Area: Checkout
Build: 2026.10.06-3    Env: staging
Browser: Safari 18 on macOS
 
Steps
  1. Add two items, apply code SAVE10
  2. Change delivery from Standard to Express
Expected
  Total keeps the 10% discount
Actual
  Discount silently removed, code field empty
 
Evidence: screen recording, HAR file
Regression test: e2e/checkout/discount.spec.ts

05 / QA STRATEGY

A strategy that grows with the product.

A three-person startup and a platform with enterprise contracts need different testing. We plan for the stage you’re at, and write down what will change the plan.

  1. 01

    Proving the idea

    One team, early users, and a product that changes every week.

    INVEST IN

    • Unit tests on rules that touch money or data
    • A CI pipeline from the first commit
    • Exploratory testing of the core journey before releases
    • A handful of smoke tests on the journey that defines the product

    NOT YET

    • Large end-to-end suites while the interface keeps moving
    • Formal load testing beyond a sanity check

    REVISIT WHENPaying customers arrive, or a second team starts contributing.

  2. 02

    Early traction

    Customers depend on it, and releases go out weekly or faster.

    INVEST IN

    • A regression test for every escaped bug
    • API and integration tests at service boundaries
    • End-to-end coverage of revenue-critical journeys
    • Gates that block merges instead of just reporting

    NOT YET

    • Long browser and device matrices until analytics justify them

    REVISIT WHENRegressions slow releases down, or a large customer is coming.

  3. 03

    Scaling

    Several teams, more traffic, and commitments to customers.

    INVEST IN

    • Load tests against a model of real traffic
    • Browser and device matrix from real usage
    • Contract tests between teams and services
    • Security-focused checks in every pipeline
    • Managed test data and preview environments

    NOT YET

    • Coverage percentages as a goal in themselves

    REVISIT WHENTeams start breaking each other, or reliability becomes contractual.

  4. 04

    Established

    Many teams, service levels, and enterprise or regulated customers.

    INVEST IN

    • Shared test tooling and conventions across teams
    • Synthetic checks of key journeys in production
    • Release metrics such as change failure rate and escaped defects
    • Independent penetration tests on a schedule, with fixes verified
WHERE EACH CHECK RUNS
TriggerFeedback targetChecks
Every commitUnder 5 minutes
  • Lint & types
  • Unit tests
  • Component tests
Every pull requestUnder 15 minutes
  • Integration
  • API & contracts
  • Smoke E2E
  • Dependency scan
Merge to mainUnder 30 minutes
  • Full E2E
  • Cross-browser
  • Performance budgets
  • Deploy & smoke
NightlyResults by morning
  • Full regression
  • Device matrix
  • ZAP baseline scan
  • Load at expected peak
Before a major releasePlanned with the release
  • Exploratory sessions
  • Load, stress & soak
  • Release readiness review

Feedback targets are what we design pipelines to meet, and we agree them with your team. Some of the work, such as exploratory sessions, will always need people.

06 / CAPABILITIES

The rest of a quality practice.

Each capability can stand alone, but production systems usually need most of them working together.

Test strategy and planning

Risk-based strategies, test plans, and clear entry and exit criteria for releases.

Test data management

Synthetic and anonymized data, factories, and seeding, so tests are repeatable and never touch real customer records.

Test environments

Preview environments per pull request, containerized dependencies, and stubs for third parties that can’t be called from tests.

Flaky test management

Detection, quarantine, and root-cause fixes, so a red build always means something.

Contract testing

Consumer-driven contracts between services, apps, and partner APIs, verified in CI before anything is deployed.

Accessibility checks

Automated axe scans in CI, plus manual keyboard and screen reader checks against WCAG 2.2 AA criteria.

Defect management

Triage, severity, and root-cause analysis that feed back into the test suite and the backlog.

Quality reporting

Test results, flaky rate, escaped defects, and feedback time, reported so people outside engineering can read them.

Release readiness

Go or no-go reviews based on evidence, with known risks, open defects, and the rollback plan stated plainly.

Team enablement

Pairing, test reviews, and written conventions, so your engineers own quality after the engagement.

Need suites built and maintained, or quality designed into a new product? Test automation · Cloud & DevOps · Custom software development · Mobile app development

07 / ENGINEERING APPROACH

Principles behind every test.

How we decide what to test, where, and how much. Testing everything equally is the same as testing nothing well.

01

Risk decides depth

Effort goes where failure would cost the most.

  • A risk map agreed with product and engineering
  • Money, data, and access paths tested hardest
  • Explicit decisions about what not to test

02

The lowest layer that can catch it

Fast, focused tests over slow, broad ones.

  • Business rules in unit tests
  • Contracts at service boundaries
  • End-to-end tests only for whole journeys

03

Fast feedback

A test nobody waits for is a test nobody skips.

  • A time budget for every pipeline stage
  • Parallel and sharded runs
  • The most relevant tests first on pull requests

04

Flaky means broken

A suite people ignore is worse than no suite.

  • Flaky tests quarantined quickly
  • Root causes fixed, not retried away
  • Flake rate tracked like any other defect

05

People explore, machines repeat

Automation checks what we know; people find what we don’t.

  • Exploratory sessions on significant features
  • Findings turned into automated checks
  • Manual scripts retired once behavior settles

06

Evidence over percentages

Coverage is a clue, not a target.

  • Release decisions based on risks covered and open
  • Escaped defects traced to the missing check
  • Reports written for the people deciding to ship

Infrastructure, pipelines, and monitoring are delivered with our cloud and DevOps practice; connections to your systems with data and integrations.

08 / TECHNOLOGIES

Tools chosen for the problem.

We recommend a stack after discovery, based on your requirements, existing platforms, and team.

Browser & end-to-end

Playwright for most new suites; Cypress or Selenium where you already use them.

PlaywrightCypressSelenium / WebDriver

Unit & component

Matched to your language and framework.

JestVitestTesting LibrarypytestJUnit

API, contract & integration

Collections people can explore, run unattended in CI.

PostmanNewmanPactREST AssuredTestcontainers

Performance & load

Scripts in the repository, reviewed like code.

k6JMeterGatlingLighthouse CI

Mobile

Native, cross-platform, and flow-based tooling.

MaestroDetoxAppiumXCUITestEspresso

Cross-browser & devices

Engines in CI; versions and real devices where they matter.

Playwright projectsSelenium GridBrowserStackSauce Labs

Security checks

Everyday checks, not a substitute for a penetration test.

OWASP ZAPSemgrepTrivygitleaks

Pipelines & reporting

In the CI you already run.

GitHub ActionsGitLab CIJenkinsAllure Report

Representative technologies; no official partnerships are implied. Explore our technology capabilities.

09 / PROCESS

How we build a quality strategy.

Seven steps from first look to a strategy that runs in your pipeline. Every step ends with something you can review.

  1. 01

    Quality assessment

    Review the product, codebase, existing tests, pipeline, defect history, and incidents. Talk to engineering, product, and support about where it hurts.

    OUTPUTAssessment of the current state, based on evidence

  2. 02

    Risk map

    Rank journeys and components by business impact and likelihood of failure: money, data, access, traffic, and how often the code changes.

    OUTPUTA risk map agreed with product and engineering

  3. 03

    Test portfolio

    Decide which risks are covered at which layer, from unit to manual, and what is deliberately left untested and why.

    OUTPUTTest strategy and a layer-by-layer plan

  4. 04

    Pipeline and gates

    Wire the checks into CI with agreed thresholds and a feedback-time budget for every stage.

    OUTPUTQuality gates running in your pipeline

  5. 05

    Riskiest coverage first

    Automate the highest-risk journeys first, set up test data and environments, and fix or quarantine what is flaky.

    OUTPUTAutomated suites for the top-risk journeys

  6. 06

    Explore and harden

    Exploratory sessions, cross-browser and device runs, performance and load tests, and security-focused checks ahead of major releases.

    OUTPUTFindings report and release readiness evidence

  7. 07

    Measure and evolve

    Track escaped defects, flaky rate, and feedback time, and revisit the risk map as the product and team grow.

    OUTPUTQuality dashboard and a regular strategy review

See how this fits into our wider delivery process.

10 / RELEVANT WORK

How we think it through.

Client case studies are published once they are approved. The work below consists of illustrative concepts, not delivered engagements or claimed results.

11 / WHY NEURALSHARKS

What working with us looks like.

Commitments you can hold us to during an engagement.

Quality in the architecture

Testability is designed in from the start: clear boundaries, injectable dependencies, and environments that can be created on demand. Tests are easier to write when the system expects them.

Custom software development

The pipeline comes first

On products we build, the pipeline and its first gates exist before the first feature, so every change from day one goes through them.

Cloud & DevOps

Tool-neutral advice

We recommend tools for your stack and your team, and keep the suite you already have when it is worth keeping. We don’t rewrite working tests for fashion.

Honest reporting

Reports say what was tested, what wasn’t, and which risks are still open. We don’t call a release safe without evidence.

Yours to keep

Suites live in your repositories and run in your pipeline, with conventions and documentation your team can maintain without us.

Clear about limits

Security-focused testing is part of everyday engineering. It doesn’t replace an independent penetration test, and we’ll tell you when you need one.

12 / FAQ

Questions to ask before you hire us.

If yours isn’t here, ask us directly. A straight answer is part of the service.

Do you replace our QA team?

Not unless that’s what you need. We can work alongside your QA engineers and developers, build a practice where none exists yet, or take on a defined area such as performance or mobile testing. The goal is a team that owns quality, not a dependency on us.

Manual or automated testing?

Both, for different jobs. Automation repeats known checks on every change. People explore, judge usability, and find the problems nobody predicted. We automate stable, high-value checks, and keep manual effort for work that needs judgment.

Playwright, Cypress, or Selenium?

For new web suites we usually recommend Playwright: it covers Chromium, Firefox, and WebKit, runs tests in parallel, and records traces on failure. Cypress suits teams already productive with it. Selenium makes sense with an existing grid, specific browser versions, or a language the others don’t support. We choose for your situation and don’t rewrite working suites.

What test coverage should we aim for?

We don’t set a single percentage. High coverage of trivial code proves little, and any number used as a target gets gamed. We aim for every high-risk journey and rule to be covered at a suitable layer, and use coverage reports to find gaps rather than to grade the team.

Can you fix our flaky, slow test suite?

Usually, yes. We measure which tests fail intermittently and why, quarantine them so the pipeline is trustworthy again, and fix the root causes: shared state, timing, test data, or environment. Then we restructure slow suites so the most useful feedback arrives first.

Do you do penetration testing or security certification?

No. Our security-focused testing covers authorization, input handling, dependency and secret scanning, and automated baseline scans in the pipeline. A penetration test by an independent specialist is a different service, and we’ll recommend one for high-risk systems. We don’t issue certifications or compliance attestations.

How do you approach performance and load testing?

We start from a model of real usage: which journeys, at what rates, with what data. Performance checks with budgets run on every change. Full load, stress, and soak tests run before major releases or expected peaks, against an environment sized like production. The result tells you what breaks first, and at what load.

Can you test mobile apps on real devices?

Yes. We choose a device and OS matrix from your analytics, run automated flows on emulators and simulators in CI, and use real devices, yours or a device cloud, for the checks that need them, such as performance, sensors, interruptions, and release candidates.

How do we get started?

With a short assessment. We review your product, existing tests, pipeline, and defect history, and talk to the people who build and support it. You get a risk map and a prioritized plan, then decide whether we implement it, you do, or we do it together.

START A PROJECT

Ship with evidence, not hope.

Tell us what you’re building, how it’s tested today, and what worries you before a release. We’ll come back with questions, an honest view of where the risk is, and a sensible first step.