Customers find the bugs first.
Regressions reach production and are reported by users or support. Every release feels like a gamble, and hotfixes crowd out planned work.
SOFTWARE TESTING
Quality engineering is a core NeuralSharks discipline. We design how a product is tested the way we design how it’s built: around its risks, its architecture, and the way it ships.
Testing at the end finds problems when they cost the most to fix. We put checks at every layer, from unit tests on business rules to load tests on the busiest journey, and wire them into the pipeline. Every change then arrives with evidence that it works.
01 / WHY TEAMS CALL US
Most teams already test. The trouble is that the testing doesn’t give them confidence, speed, or either.
Regressions reach production and are reported by users or support. Every release feels like a gamble, and hotfixes crowd out planned work.
Days of clicking through a spreadsheet of test cases before each release. It’s slow, it misses things, and it grows with every feature.
Flaky tests fail at random, so people rerun them until they pass. Real failures get ignored along with the noise.
The pipeline is so slow that engineers batch their changes and learn about breakages long after they’ve moved on to something else.
Coverage is a percentage on a dashboard. Nobody can say which journeys are protected, or which risks were accepted on purpose.
Performance, scale, and security are checked once before launch, if at all. The first real traffic peak becomes the load test.
02 / CI PIPELINE
Each stage runs the cheapest checks that can catch a fault at that level, and each has a gate. Choose a run to see what a gate stops, and select any stage to see what it checks.
feat(checkout): add discount codes
A pull request adds discount codes to checkout. Every stage passes, so it reaches customers with the evidence attached to the pull request.
IN PROGRESSRunning Commit: stage 1 of 7.
01Running
GATEHooks pass and a second engineer reviews the change
$ git push origin feat/discount-codes
lint-staged · prettier, eslint
✓ 14 files formatted and linted
✓ pull request opened, review requested
An illustrative pipeline with representative timings and results. We size the stages to your product: a young product may need only a few of these gates, and that is part of the strategy.
03 / WHAT WE TEST
The kinds of products we test most often, and the part of each that usually needs the most care.
Multi-tenant products with roles, subscriptions, admin tools, and integrations.
THE HARD PARTProving one tenant can never see another’s data, in every screen, export, search result, and API.
Customer-facing sites and applications used across browsers, devices, and connection speeds.
THE HARD PARTBehavior that changes with the browser engine, the viewport, and the network, not with the code.
iOS and Android apps, native or cross-platform, together with the APIs behind them.
THE HARD PARTDevice and OS fragmentation, interrupted sessions, poor connectivity, and users who never update.
Public, partner, and internal APIs, webhooks, and connections to third-party systems.
THE HARD PARTConsumers you can’t see, and partner sandboxes that behave differently from production.
Checkout, subscriptions, invoices, refunds, and the reconciliation behind them.
THE HARD PARTRounding, currencies, proration, and partial refunds, where nearly right is still wrong.
Queues, workers, real-time updates, and the processes that react to events.
THE HARD PARTDuplicate, delayed, and out-of-order messages, which usually appear only under load or failure.
Assistants, search, classification, and generation built into a product.
THE HARD PARTAnswers that vary from run to run, so pass and fail need evaluation sets rather than exact matches.
Products with years of behavior, few tests, and changes coming.
THE HARD PARTCapturing what the system actually does today, before anyone changes it.
04 / TESTING DISCIPLINES
Each kind of testing answers a different question, at a different cost. A strategy decides which questions your product needs answered, and how often.
01 / MANUAL QA · THE QUESTION IT ANSWERS
People still find what automation can’t: confusing flows, wrong copy, visual defects, and problems nobody thought to write a test for. We run exploratory sessions against written charters, test new features by hand while they are still changing, and write bug reports a developer can act on without a follow-up conversation.
YOU RECEIVEExploratory session notes, manual test cases, and actionable bug reports
# BUG-0142 Discount removed when delivery changes
Severity: High Area: Checkout
Build: 2026.10.06-3 Env: staging
Browser: Safari 18 on macOS
Steps
1. Add two items, apply code SAVE10
2. Change delivery from Standard to Express
Expected
Total keeps the 10% discount
Actual
Discount silently removed, code field empty
Evidence: screen recording, HAR file
Regression test: e2e/checkout/discount.spec.ts
02 / FUNCTIONAL TESTING · THE QUESTION IT ANSWERS
We turn requirements into acceptance criteria before development starts, then test against them with boundary values, equivalence classes, and decision tables. Ambiguity found while writing a test is cheaper to resolve than ambiguity found in production.
YOU RECEIVEAcceptance criteria and functional tests traceable to requirements
Feature: Discount codes at checkout
Rule: Percentage codes need a minimum spend of $50
Scenario Outline: Minimum spend is respected
Given a cart worth <subtotal>
When the customer applies "SAVE10"
Then the discount is <discount>
Examples:
| subtotal | discount | why |
| $49.99 | $0.00 | just below |
| $50.00 | $5.00 | exactly minimum |
| $120.00 | $12.00 | normal |
03 / INTEGRATION TESTING · THE QUESTION IT ANSWERS
Mocks agree with whatever you tell them. Integration tests run your code against real dependencies in disposable containers, so constraints, transactions, serialization, and migrations are tested as they will behave in production.
YOU RECEIVEIntegration suites running against real dependencies in CI
import { PostgreSqlContainer, type StartedPostgreSqlContainer }
from "@testcontainers/postgresql";
let db: StartedPostgreSqlContainer;
beforeAll(async () => {
db = await new PostgreSqlContainer("postgres:16").start();
await migrate(db.getConnectionUri());
}, 60_000);
afterAll(() => db.stop());
it("rejects a repeated idempotency key", async () => {
const orders = new OrderRepository(db.getConnectionUri());
await orders.create(anOrder({ idempotencyKey: "k-1" }));
await expect(orders.create(anOrder({ idempotencyKey: "k-1" })))
.rejects.toThrow(DuplicateOrder);
});
04 / API TESTING · THE QUESTION IT ANSWERS
We test status codes, schemas, headers, authentication, pagination, and error handling for every endpoint, along with the unhappy paths clients actually hit. Postman collections are easy for the whole team to explore, and Newman runs the same collections in CI.
YOU RECEIVEAPI test collections, schema checks, and Newman runs in the pipeline
// Postman test script for POST /v1/orders
pm.test("201 with a Location header", () => {
pm.response.to.have.status(201);
pm.response.to.have.header("Location");
});
pm.test("matches the Order schema", () => {
const schema = JSON.parse(pm.collectionVariables.get("orderSchema"));
pm.response.to.have.jsonSchema(schema);
});
pm.test("a replayed key returns the same order", () => {
pm.expect(pm.response.json().id)
.to.eql(pm.environment.get("firstOrderId"));
});
// CI: newman run orders.postman_collection.json -e ci.json
05 / REGRESSION TESTING · THE QUESTION IT ANSWERS
Every escaped bug gets a test that would have caught it, tagged by area. Pull requests run the regression tests for the areas they touch; the full suite runs nightly and before every release, so the safety net grows with the product without slowing every change.
YOU RECEIVEA tagged regression suite that grows with every fixed defect
test("keeps the discount when delivery changes", {
tag: ["@regression", "@checkout"],
annotation: { type: "issue", description: "BUG-0142" },
}, async ({ page }) => {
await page.goto("/checkout?cart=two-items");
await page.getByLabel("Discount code").fill("SAVE10");
await page.getByRole("button", { name: "Apply" }).click();
await page.getByRole("radio", { name: "Express" }).check();
await expect(page.getByTestId("order-total")).toHaveText("$94.50");
});
// Pull request: npx playwright test --grep @checkout
// Nightly: npx playwright test --grep @regression
06 / AUTOMATION TESTING · THE QUESTION IT ANSWERS
Automation is software, so we engineer it like software. Fixtures create their own data through APIs, tests run in parallel without sharing state, and failures produce traces rather than mysteries. A flaky test is treated as a defect, not a nuisance.
YOU RECEIVEA maintainable automation framework, conventions, and CI integration
Test automationimport { test as base } from "@playwright/test";
export const test = base.extend<{
customer: Customer; checkout: CheckoutPage;
}>({
// Each test creates its own customer through the API:
// no shared state, safe in parallel, cleaned up after.
customer: async ({ request }, use) => {
const customer = await createCustomer(request, { plan: "pro" });
await use(customer);
await deleteCustomer(request, customer.id);
},
checkout: async ({ page, customer }, use) => {
await signInAs(page, customer);
await use(new CheckoutPage(page));
},
});
07 / END-TO-END TESTING · THE QUESTION IT ANSWERS
A small number of high-value journeys, run through the real interface and backend in a production-like environment. We usually recommend Playwright for new suites, and work with Cypress or Selenium where a team already relies on them. Locators follow what users see, not CSS that changes weekly.
YOU RECEIVEEnd-to-end coverage of critical journeys, with traces on failure
import { test, expect } from "./fixtures";
test("customer buys with a saved card", async ({ page, checkout }) => {
await checkout.addToCart("Trail Runner 2", { size: "42" });
await checkout.open();
await checkout.payWithSavedCard();
await expect(page.getByRole("heading", { name: "Order confirmed" }))
.toBeVisible();
await expect(page.getByText(/Order #\d+/)).toBeVisible();
});
// On failure the report includes a trace, screenshot, and video.
08 / PERFORMANCE TESTING · THE QUESTION IT ANSWERS
Performance is a requirement with numbers, not an impression. We agree budgets for pages and endpoints, check them on every change, and compare against the main branch, so slowdowns are caught in the change that caused them rather than months later.
YOU RECEIVEPerformance budgets, baseline comparisons, and trend reports
{
"ci": {
"collect": {
"url": ["https://preview.example.com/checkout"],
"numberOfRuns": 3
},
"assert": {
"assertions": {
"largest-contentful-paint": ["error", { "maxNumericValue": 2500 }],
"cumulative-layout-shift": ["error", { "maxNumericValue": 0.1 }],
"total-blocking-time": ["warn", { "maxNumericValue": 200 }],
"resource-summary:script:size": ["error", { "maxNumericValue": 300000 }]
}
}
}
}
09 / LOAD TESTING · THE QUESTION IT ANSWERS
We build a load model from real traffic, run it against an environment sized like production, and push past the expected peak to find the first bottleneck. Scripts live in the repository and are reviewed like code. We use k6 by default, and JMeter or Gatling where protocols or team skills call for them.
YOU RECEIVELoad model, repeatable load tests, and a capacity findings report
import http from "k6/http";
import { check } from "k6";
export const options = {
scenarios: { sale_peak: {
executor: "ramping-arrival-rate",
startRate: 20, timeUnit: "1s",
preAllocatedVUs: 200, maxVUs: 1000,
stages: [
{ target: 200, duration: "10m" }, // expected peak
{ target: 600, duration: "10m" }, // 3x headroom
{ target: 600, duration: "20m" }, // hold
],
} },
thresholds: {
"http_req_duration{name:checkout}": ["p(95)<400"],
http_req_failed: ["rate<0.005"],
},
};
export default function () {
const res = http.post(`${__ENV.BASE_URL}/v1/checkout`, cart(),
{ tags: { name: "checkout" } });
check(res, { "created": r => r.status === 201 });
}
10 / CROSS-BROWSER TESTING · THE QUESTION IT ANSWERS
The browser matrix comes from your analytics, not from habit. Chromium, Firefox, and WebKit run in CI on every change; specific browser versions and real devices are covered through Selenium Grid or a cloud device service where they matter.
YOU RECEIVEA usage-based browser matrix running in CI
export default defineConfig({
// Matrix from the last 90 days of analytics.
projects: [
{ name: "chrome", use: { ...devices["Desktop Chrome"] } },
{ name: "edge", use: { ...devices["Desktop Edge"], channel: "msedge" } },
{ name: "firefox", use: { ...devices["Desktop Firefox"] } },
{ name: "safari", use: { ...devices["Desktop Safari"] } },
{ name: "iphone", use: { ...devices["iPhone 15"] } },
{ name: "android", use: { ...devices["Pixel 7"] } },
],
use: { trace: "retain-on-failure", screenshot: "only-on-failure" },
});
11 / MOBILE TESTING · THE QUESTION IT ANSWERS
Mobile adds what the web mostly avoids: device fragmentation, permissions, interruptions, offline use, and app-store releases. Automated flows run on emulators and simulators in CI; real devices cover performance, sensors, and release candidates.
YOU RECEIVEDevice matrix, automated mobile flows, and release-candidate checks
appId: com.example.fieldapp
---
- launchApp:
clearState: true
- runFlow: flows/sign-in.yaml
- setAirplaneMode: enabled # Android
- tapOn: "Today's jobs"
- tapOn: "Check in"
- assertVisible: "Saved. Will sync when online"
- setAirplaneMode: disabled
- extendedWaitUntil:
visible: "Synced"
timeout: 15000
12 / SECURITY-FOCUSED TESTING · THE QUESTION IT ANSWERS
Security checks that belong in everyday engineering: authorization tested for every role, input handling, dependency and secret scanning, and an automated baseline scan of staging. It complements an independent penetration test rather than replacing one.
YOU RECEIVEAuthorization test suites and security checks in every pipeline
// Generated from the permission matrix: every role, every rule.
const cases = [
{ role: "customer", path: "/v1/orders/:own", status: 200 },
{ role: "customer", path: "/v1/orders/:other", status: 404 },
{ role: "support", path: "/v1/orders/:other", status: 200 },
{ role: "support", path: "/v1/refunds", status: 403, method: "post" },
{ role: "anonymous", path: "/v1/orders/:own", status: 401 },
];
test.each(cases)("$role $path → $status",
async ({ role, path, status, method = "get" }) => {
const res = await as(role)[method](resolve(path));
expect(res.status).toBe(status);
});
// Pipeline: zap-baseline.py · npm audit · trivy · gitleaks
13 / CI/CD QUALITY GATES · THE QUESTION IT ANSWERS
Every check above is only useful if it can stop a release. Gates are required status checks with agreed thresholds, ordered so the fastest feedback comes first, and each gate has a feedback-time budget so the pipeline stays fast enough that nobody routes around it.
YOU RECEIVEPipeline gates with agreed thresholds and feedback-time budgets
Cloud & DevOps# Excerpt: each job is a required status check.
jobs:
unit:
steps:
- run: npx jest --ci --coverage
integration:
needs: unit
steps:
- run: npx jest -c jest.integration.config.ts
- run: npx newman run api/orders.postman_collection.json
e2e:
needs: integration
strategy:
matrix: { browser: [chromium, firefox, webkit] }
steps:
- run: npx playwright test --project=${{ matrix.browser }}
performance:
needs: e2e
steps:
- run: k6 run perf/checkout-smoke.js
deploy:
needs: [e2e, performance]
if: github.ref == 'refs/heads/main'
05 / QA STRATEGY
A three-person startup and a platform with enterprise contracts need different testing. We plan for the stage you’re at, and write down what will change the plan.
01
One team, early users, and a product that changes every week.
INVEST IN
NOT YET
REVISIT WHENPaying customers arrive, or a second team starts contributing.
02
Customers depend on it, and releases go out weekly or faster.
INVEST IN
NOT YET
REVISIT WHENRegressions slow releases down, or a large customer is coming.
03
Several teams, more traffic, and commitments to customers.
INVEST IN
NOT YET
REVISIT WHENTeams start breaking each other, or reliability becomes contractual.
04
Many teams, service levels, and enterprise or regulated customers.
INVEST IN
| Trigger | Feedback target | Checks |
|---|---|---|
| Every commit | Under 5 minutes | |
| Every pull request | Under 15 minutes | |
| Merge to main | Under 30 minutes | |
| Nightly | Results by morning | |
| Before a major release | Planned with the release |
Feedback targets are what we design pipelines to meet, and we agree them with your team. Some of the work, such as exploratory sessions, will always need people.
06 / CAPABILITIES
Each capability can stand alone, but production systems usually need most of them working together.
Risk-based strategies, test plans, and clear entry and exit criteria for releases.
Synthetic and anonymized data, factories, and seeding, so tests are repeatable and never touch real customer records.
Preview environments per pull request, containerized dependencies, and stubs for third parties that can’t be called from tests.
Detection, quarantine, and root-cause fixes, so a red build always means something.
Consumer-driven contracts between services, apps, and partner APIs, verified in CI before anything is deployed.
Automated axe scans in CI, plus manual keyboard and screen reader checks against WCAG 2.2 AA criteria.
Triage, severity, and root-cause analysis that feed back into the test suite and the backlog.
Test results, flaky rate, escaped defects, and feedback time, reported so people outside engineering can read them.
Go or no-go reviews based on evidence, with known risks, open defects, and the rollback plan stated plainly.
Pairing, test reviews, and written conventions, so your engineers own quality after the engagement.
Need suites built and maintained, or quality designed into a new product? Test automation · Cloud & DevOps · Custom software development · Mobile app development
07 / ENGINEERING APPROACH
How we decide what to test, where, and how much. Testing everything equally is the same as testing nothing well.
01
Effort goes where failure would cost the most.
02
Fast, focused tests over slow, broad ones.
03
A test nobody waits for is a test nobody skips.
04
A suite people ignore is worse than no suite.
05
Automation checks what we know; people find what we don’t.
06
Coverage is a clue, not a target.
Infrastructure, pipelines, and monitoring are delivered with our cloud and DevOps practice; connections to your systems with data and integrations.
08 / TECHNOLOGIES
We recommend a stack after discovery, based on your requirements, existing platforms, and team.
Playwright for most new suites; Cypress or Selenium where you already use them.
Matched to your language and framework.
Collections people can explore, run unattended in CI.
Scripts in the repository, reviewed like code.
Native, cross-platform, and flow-based tooling.
Engines in CI; versions and real devices where they matter.
Everyday checks, not a substitute for a penetration test.
In the CI you already run.
Representative technologies; no official partnerships are implied. Explore our technology capabilities.
09 / PROCESS
Seven steps from first look to a strategy that runs in your pipeline. Every step ends with something you can review.
Review the product, codebase, existing tests, pipeline, defect history, and incidents. Talk to engineering, product, and support about where it hurts.
OUTPUTAssessment of the current state, based on evidence
Rank journeys and components by business impact and likelihood of failure: money, data, access, traffic, and how often the code changes.
OUTPUTA risk map agreed with product and engineering
Decide which risks are covered at which layer, from unit to manual, and what is deliberately left untested and why.
OUTPUTTest strategy and a layer-by-layer plan
Wire the checks into CI with agreed thresholds and a feedback-time budget for every stage.
OUTPUTQuality gates running in your pipeline
Automate the highest-risk journeys first, set up test data and environments, and fix or quarantine what is flaky.
OUTPUTAutomated suites for the top-risk journeys
Exploratory sessions, cross-browser and device runs, performance and load tests, and security-focused checks ahead of major releases.
OUTPUTFindings report and release readiness evidence
Track escaped defects, flaky rate, and feedback time, and revisit the risk map as the product and team grow.
OUTPUTQuality dashboard and a regular strategy review
See how this fits into our wider delivery process.
10 / RELEVANT WORK
Client case studies are published once they are approved. The work below consists of illustrative concepts, not delivered engagements or claimed results.
A design study for coordinated, observable workflows across the business systems an operations team depends on.
A design study for a headless storefront that stays fast and accessible under campaign traffic, with releases gated by evidence.
11 / WHY NEURALSHARKS
Commitments you can hold us to during an engagement.
Testability is designed in from the start: clear boundaries, injectable dependencies, and environments that can be created on demand. Tests are easier to write when the system expects them.
Custom software developmentOn products we build, the pipeline and its first gates exist before the first feature, so every change from day one goes through them.
Cloud & DevOpsWe recommend tools for your stack and your team, and keep the suite you already have when it is worth keeping. We don’t rewrite working tests for fashion.
Reports say what was tested, what wasn’t, and which risks are still open. We don’t call a release safe without evidence.
Suites live in your repositories and run in your pipeline, with conventions and documentation your team can maintain without us.
Security-focused testing is part of everyday engineering. It doesn’t replace an independent penetration test, and we’ll tell you when you need one.
12 / FAQ
If yours isn’t here, ask us directly. A straight answer is part of the service.
Not unless that’s what you need. We can work alongside your QA engineers and developers, build a practice where none exists yet, or take on a defined area such as performance or mobile testing. The goal is a team that owns quality, not a dependency on us.
Both, for different jobs. Automation repeats known checks on every change. People explore, judge usability, and find the problems nobody predicted. We automate stable, high-value checks, and keep manual effort for work that needs judgment.
For new web suites we usually recommend Playwright: it covers Chromium, Firefox, and WebKit, runs tests in parallel, and records traces on failure. Cypress suits teams already productive with it. Selenium makes sense with an existing grid, specific browser versions, or a language the others don’t support. We choose for your situation and don’t rewrite working suites.
We don’t set a single percentage. High coverage of trivial code proves little, and any number used as a target gets gamed. We aim for every high-risk journey and rule to be covered at a suitable layer, and use coverage reports to find gaps rather than to grade the team.
Usually, yes. We measure which tests fail intermittently and why, quarantine them so the pipeline is trustworthy again, and fix the root causes: shared state, timing, test data, or environment. Then we restructure slow suites so the most useful feedback arrives first.
No. Our security-focused testing covers authorization, input handling, dependency and secret scanning, and automated baseline scans in the pipeline. A penetration test by an independent specialist is a different service, and we’ll recommend one for high-risk systems. We don’t issue certifications or compliance attestations.
We start from a model of real usage: which journeys, at what rates, with what data. Performance checks with budgets run on every change. Full load, stress, and soak tests run before major releases or expected peaks, against an environment sized like production. The result tells you what breaks first, and at what load.
Yes. We choose a device and OS matrix from your analytics, run automated flows on emulators and simulators in CI, and use real devices, yours or a device cloud, for the checks that need them, such as performance, sensors, interruptions, and release candidates.
With a short assessment. We review your product, existing tests, pipeline, and defect history, and talk to the people who build and support it. You get a risk map and a prioritized plan, then decide whether we implement it, you do, or we do it together.
START A PROJECT
Tell us what you’re building, how it’s tested today, and what worries you before a release. We’ll come back with questions, an honest view of where the risk is, and a sensible first step.