Skip to main content

DevSecOps research

The governance problem nobody wanted to solve​

Most DevSecOps transformations start with tools. A SAST scanner goes into the pipeline and the programme declares progress. Six months later, thousands of findings remain unresolved, developers ignore the noise, and the security team has more work than before. The same pattern appeared at the start of a Tier-1 bank transformation.

The problem was never the tools. It was governance, specifically the absence of a research-informed model that could answer three questions: where are we today, where do we need to be, and what is the most effective path between those two points. The transformation was built on a maturity model grounded in the research of Forsgren, Humble, and Kim, mapped to the ASD Information Security Manual (ISM) and ISO 27001 controls, and measured through DORA metrics. Every team was assessed against the model quarterly. Every initiative was prioritized by its expected impact on both delivery performance and security posture, and every investment was justified with data. That approach is what separated this transformation from the tool-first failures observed elsewhere. Governance, here, meant knowing what worked, measuring whether it worked, and adjusting when it did not.

Overview​

DevSecOps integrates security into the DevOps process so that security becomes a shared responsibility across the software development lifecycle. It automates security checks at every phase, from initial design through integration, testing, deployment, and delivery, and it depends on collaboration between development, security, and operations teams. Integrating security early reduces vulnerabilities and improves the security posture of the applications that ship.

DevSecOps research draws from several disciplines: software engineering, information security, organizational psychology, and systems thinking. The empirical work of Forsgren et al. in "Accelerate" (2018) demonstrated that security integration does not slow delivery. Teams that integrate security into their daily work achieve higher deployment frequency and lower change failure rates than teams that treat security as an external gate, a finding that contradicts decades of received wisdom.

Research areas​

  • Continuous integration/continuous deployment (CI/CD): automating the integration and deployment of code changes so releases are reliable and frequent.
  • Security automation: using automated tools and processes to identify and mitigate vulnerabilities in the development lifecycle.
  • Compliance and governance: keeping development practices compliant with standards and regulations, and enforcing security policies throughout development.
  • Threat modeling: identifying and addressing potential threats during the design phase.
  • DevSecOps culture: shared responsibility for security across all teams.
  • Maturity assessment and adoption pathways: measuring organizational maturity and defining evidence-based paths for improvement.

Maturity models and adoption paths​

Effective transformation requires knowing where the organization stands and having a research-informed roadmap for getting better. At the bank, a five-level maturity model was developed, inspired by the ASD ISM's maturity tiers and the ACSC Essential Eight maturity model, and informed by the DORA research:

  • Level 1 (Reactive): security is performed manually and after the fact, with no automation. Compliance is demonstrated through periodic audits.
  • Level 2 (Managed): basic security scanning is integrated into CI/CD pipelines. Findings are tracked, but remediation is ad hoc.
  • Level 3 (Defined): security policies are codified and enforced automatically, and threat modeling is performed for new services. Metrics are collected but not yet used to drive decisions.
  • Level 4 (Measured): DORA metrics and security KPIs are tracked continuously. Teams use data to prioritize security work, and compliance evidence is generated automatically.
  • Level 5 (Optimized): security is embedded in the developer experience through golden paths, predictive analytics identify emerging risks, and improvement is driven by data.

The adoption pathway is not linear. Teams within the same organization sit at different levels, and that is expected, so the governance model has to account for it. At the bank, maturity was tracked by team and by capability, which made it possible to target investments where impact would be highest. A team at Level 2 for container security but Level 4 for SAST does not need a new SAST tool. It needs container security training and guardrails.

Continuous integration/continuous deployment (CI/CD)​

CI/CD automates the integration and deployment of code changes so releases are reliable and frequent. The core practices are automated testing, frequent merging of code changes, automatic deployment to production, and infrastructure as code, where computing infrastructure is managed through machine-readable definition files.

Automated testing​

In banking environments, automated testing extends past functional correctness to regulatory validation. Key techniques include unit, integration, and end-to-end testing at the component and application-flow level; test automation frameworks such as Selenium, JUnit, and TestNG; and security test suites, which are automated tests that validate security controls including authentication, authorization, input validation, and encryption.

At the bank, the team maintained security test suites that validated OWASP Top 10 controls for every API endpoint: injection prevention, authentication enforcement, sensitive data exposure, and access control. These tests ran on every pull request and were required for merge approval.

Continuous integration​

Continuous integration merges code changes frequently so issues surface early. It relies on version control, build automation, code reviews, and fast feedback to developers on quality and security problems.

Continuous deployment​

Continuous deployment pushes code changes automatically to production. It requires deployment pipelines, rollback mechanisms for when issues are detected, monitoring and logging, and blue-green deployments, which run two identical production environments to reduce downtime and risk.

In regulated financial services, continuous deployment still has to be controlled: every deployment must be automated, auditable, and reversible. At the bank, the team used canary deployments for critical banking services. New versions went out to a small percentage of traffic first, were monitored for functional and security anomalies, and reached full production only after passing automated health checks. Rollback was automatic and took under sixty seconds.

Security automation​

Security automation uses tools and processes to detect and mitigate vulnerabilities automatically. The main techniques are static application security testing (SAST), dynamic application security testing (DAST), container security, and security orchestration, automation, and response (SOAR), which integrates security tools so that threat detection and response can be automated end to end.

Static application security testing (SAST)​

SAST analyzes source code for vulnerabilities. It combines code analysis tools, security-focused code reviews, secure coding practices, and automated code review tools such as SonarQube and Checkmarx.

The research on SAST effectiveness is consistent: it catches a meaningful percentage of vulnerabilities early but generates significant false positive noise. At the bank, this was addressed by tuning SAST rules to focus on the vulnerability classes most relevant to the bank's actual threats (injection flaws, authentication bypasses, and sensitive data exposure in APIs) and suppressing rules that produced noise without meaningful risk reduction. False positive rates dropped from over forty percent to under ten percent, and developer trust in the tooling increased accordingly.

Applying AI to security automation is an active and rapidly expanding research area. Fu, Pasuksmit, and Tantithamthavorn (2024) systematically analysed 99 papers from 2017 to 2023 and identified 12 distinct security tasks across the DevSecOps lifecycle that AI can address, from vulnerability detection and malware classification to security policy generation and compliance verification. Their evaluation of 65 benchmarks found significant variation in AI effectiveness across these tasks: vulnerability detection is the most mature, and automated remediation has the most room for improvement. Their survey reinforces what the bank found in practice: AI driven security tools are most effective when narrowly scoped to specific, well-defined tasks rather than deployed as general-purpose security oracles.

Dynamic application security testing (DAST)​

DAST tests running applications for vulnerabilities. Techniques include penetration testing, which simulates attacks to find weaknesses; vulnerability scanning against known issues; other automated security testing tools; and interactive application security testing (IAST), which combines elements of SAST and DAST for real time detection.

Container security​

Container security covers the ways containerized applications can be kept secure: scanning container images for vulnerabilities, monitoring containers at runtime for security issues, securing network communication between containers, and applying Kubernetes security best practices to clusters.

Compliance and governance​

Compliance work covers policy enforcement across the development process, audits against regulations, risk management, and data privacy obligations under regulations such as GDPR and the Australian Privacy Act 1988 (Australian Privacy Principles).

The ASD Information Security Manual (ISM) provides a comprehensive structure for organizing these activities in Australian regulated environments, with controls spanning governance, physical security, personnel security, and ICT security. Complementing it, the ACSC Essential Eight defines eight prioritised mitigation strategies that map cleanly to DevSecOps pipeline stages. At the bank, this mapping was built explicitly: asset discovery and classification happened during service registration, security scanning and patching (aligned to Essential Eight application patching and OS patching controls) happened during build and deploy, runtime monitoring ran continuously in production, incident response playbooks were triggered automatically by SIEM alerts, and automated rollback and disaster recovery were tested quarterly through chaos engineering exercises.

ISO 27001 provided the management system framework that wrapped around these technical controls. The organization maintained its ISO 27001 certification continuously rather than preparing for it annually, because automated compliance evidence generation meant the evidence was always current.

Policy enforcement​

Policy enforcement means defining security policies for the development process and then automating their enforcement, monitoring compliance with them, and running automated compliance checks against them.

Audit and compliance​

Audit and compliance covers regulatory compliance, security audits that identify vulnerabilities, compliance reporting to stakeholders, and continuous compliance backed by continuous monitoring and auditing.

Risk management​

Risk management covers risk assessment in the development process, mitigation measures, incident response for security incidents and breaches, and threat intelligence for identifying and mitigating emerging threats.

Threat modelling​

Threat modeling identifies and addresses potential security threats during the design phase. The core steps are identifying the assets that need protection, identifying threats to those assets, assessing the vulnerabilities those threats could exploit, and implementing measures to mitigate the risks.

The STRIDE methodology (Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, Elevation of Privilege), originally developed at Microsoft, provides a systematic framework for identifying threat categories. The DREAD model (Damage, Reproducibility, Exploitability, Affected Users, Discoverability) provides a complementary framework for prioritizing threats by severity.

At the bank, STRIDE-based threat models were required for every new service and every significant architectural change. The threat model was a required artifact in the design review process, stored alongside the architecture decision records in the same repository as the code, so the models were version-controlled, reviewable, and traceable to specific code changes. DREAD scoring prioritized remediation work in the backlog, which put the highest-risk threats first in the queue.

Identifying assets​

Asset identification starts with an inventory of everything that needs protection, then classifies assets by importance and sensitivity and values them to determine the level of protection required.

In banking, asset classification follows a strict taxonomy: customer PII, financial transaction data, authentication credentials, and regulatory reporting data each have specific handling requirements defined by regulation. The asset inventory must be automated and continuously reconciled against the live environment. At the bank, service mesh telemetry was built into the asset discovery process so that every API, database, and message queue was automatically discovered, classified, and registered in the configuration management database.

Identifying threats​

Threat identification draws on threat modeling frameworks such as STRIDE and DREAD, threat intelligence feeds for emerging threats, and attack surface analysis.

Assessing vulnerabilities​

Vulnerability assessment combines vulnerability scanning of the system, penetration testing, and security audits.

Mitigating risks​

Risk mitigation means developing mitigation strategies, implementing security controls to address the risks, and continuously monitoring the system so new risks are caught and addressed as they appear.

DevSecOps culture​

DevSecOps culture spreads responsibility for security across all teams. The practices are security training for all team members, collaboration between development, security, and operations, security champions within each team, and continuous improvement based on feedback and lessons learned.

The research of Forsgren et al. in "Accelerate" shows culture is not a soft, unmeasurable concept. It is a statistically significant predictor of both delivery performance and security outcomes. Organizations with generative cultures (high cooperation, low blame, emphasis on learning) outperform bureaucratic and pathological cultures on every DORA metric. That finding shaped the cultural dimension of the bank's transformation: culture was measured through team surveys and tracked as a leading indicator, and interventions were made when teams showed blame-oriented behaviors that suppressed incident reporting and learning.

Security training​

Security training spans awareness training that raises familiarity with security issues, role-based training tailored to each team member's responsibilities, and ongoing training that keeps people current with security practices.

Collaboration​

Collaboration means cross-functional teams that include members from development, security, and operations, shared security goals that all teams are responsible for achieving, and open channels for sharing information and working security issues together.

Security champions​

The security champions practice embeds one champion in each team to promote security within it. Champions are selected from team members who are already keen on security, given advanced training, and supported over time so they can help their own teams.

At the bank, the security champions program was the single most effective cultural investment made. Every delivery team embedded one champion: a developer who received advanced security training, participated in a monthly community of practice, and served as the first point of contact for security questions within their team. Champions were enablers rather than gatekeepers. They helped their teams interpret SAST findings, write threat models, and design secure architectures. Within a year, the program reduced security review escalations by over fifty percent, because teams were resolving common security questions internally instead of waiting in queue for the central security team.

Continuous improvement​

Continuous improvement runs on feedback loops that gather input on security practices, analysis of lessons learned from incidents and breaches, and continuous monitoring of the practices themselves for areas to improve.

AI-driven research methods​

AI can also support the research process itself. Relevant methods include automated literature review, predictive modeling of research outcomes and trends, data-driven hypothesis generation, and AI-enhanced experimentation, where AI driven tools optimize experimental design and analysis.

The intersection of AI and software vulnerability management is producing research with immediate practical implications. A 2026 ICSE study surveying 60 experienced security professionals found that practitioners praise AI analytics for 'filtering out noise and focusing on real threats', the same noise-reduction problem that drove SAST tuning at the bank. The study also found practitioners cautious about AI driven remediation, preferring AI that identifies and prioritises vulnerabilities while leaving remediation decisions to human engineers. That human-in-the-loop preference matches the governance model at the bank, where AI tools informed decisions but did not execute changes without review.

AI-driven research tools​

  • Claude Code with web search and MCP: for synthesising primary sources directly, reading arXiv papers, CVE advisories, and vendor security bulletins in context rather than working from a pre-indexed abstract, and citing exactly what it read. This site's own nightly arXiv research sweep and Tech Leaders Brief are both built this way, not on a third-party research API.
  • Semantic Scholar: an AI powered research tool that helps researchers find relevant papers and summarize key findings.
  • Anthropic's own research publications: Anthropic publishes its safety and alignment research directly, relevant reading for anyone assessing an AI vendor's own security posture before adopting its tools.
  • Allen Institute for AI (AI2): an institute dedicated to AI research and its applications in various fields.

DORA metrics in DevSecOps research​

DORA (DevOps Research and Assessment) metrics measure the performance and effectiveness of DevSecOps practices. The four are:

  • Deployment frequency: how often new code is deployed to production.
  • Lead Time for Changes: how long a code change takes to go from commit to production.
  • Change Failure Rate: the percentage of changes that result in a failure in production.
  • Mean Time to Restore (MTTR): the average time it takes to restore service after a failure.

The Accelerate research shows the connection between DORA metrics and DevSecOps maturity is causal, not merely correlational. Structural equation modeling links specific technical practices (continuous integration, trunk-based development, automated testing, security integration) to improvements in DORA metrics, which in turn drive improvements in organizational performance. DevSecOps research is the empirical foundation for investment decisions, not an academic exercise.

Examples of applying DORA metrics in DevSecOps research projects​

  1. Deployment frequency: increasing it lets teams iterate on improvements and deliver features to users faster, for example moving from monthly to weekly releases. At the bank, deployment frequency was a leading indicator of golden path adoption. Teams that moved to the golden path increased deployment frequency by three to five times within the first quarter.

  2. Lead Time for Changes: reducing it lets teams respond faster to new requirements and issues, and automating the CI/CD pipeline cuts the time it takes to deploy updates. Lead time was decomposed into coding time, review time, security scan time, and deployment time, so the bottleneck stages could be identified and improved precisely.

  3. Change Failure Rate: monitoring and reducing it keeps updates from damaging production, and testing and validation catch issues before they reach it. Failures were categorized by root cause (functional defect, security vulnerability, configuration error, dependency failure) to show which prevention strategies were most effective.

  4. Mean Time to Restore (MTTR): minimizing it limits downtime, and automated rollback mechanisms restore service quickly after a failure. MTTR for security incidents was tracked separately from MTTR for operational incidents, and both were reported quarterly to the board-level technology risk committee.

References​

  1. Forsgren, N., Humble, J., and Kim, G. Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations. IT Revolution Press, 2018.

  2. Australian Government Information Security Manual (ISM). Australian Signals Directorate. Available at: https://www.cyber.gov.au/resources-business-and-government/essential-cyber-security/ism

  3. ISO/IEC 27001:2022 -- Information security, cybersecurity and privacy protection -- Information security management systems -- Requirements. International Organization for Standardization, 2022.

  4. Shostack, A. Threat Modeling: Designing for Security. Wiley, 2014. (Comprehensive treatment of STRIDE and DREAD methodologies.)

  5. Howard, M. and Lipner, S. The Security Development Lifecycle: SDL: A Process for Developing Demonstrably More Secure Software. Microsoft Press, 2006. (Original source of the STRIDE threat modeling framework.)

  6. ACSC Essential Eight Maturity Model. Australian Cyber Security Centre, 2023. Available at: https://www.cyber.gov.au/resources-business-and-government/essential-cyber-security/essential-eight

  7. Forsgren, N. and Kersten, M. "DevOps Metrics." Communications of the ACM, Vol. 61, No. 4, April 2018, pages 44–48. https://doi.org/10.1145/3159169

  8. Kim, G., Humble, J., Debois, P., and Willis, J. The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations. IT Revolution Press, 2016.

  9. Fu, M., Pasuksmit, J., & Tantithamthavorn, C. (2024). "AI for DevSecOps: A Landscape and Future Opportunities." arXiv:2404.04839. https://arxiv.org/abs/2404.04839

  10. Kholoosi, M. M., Le, T. H. M., & Babar, M. A. (2026). "Software Vulnerability Management in the Era of Artificial Intelligence: An Industry Perspective." IEEE/ACM ICSE 2026. arXiv:2512.18261. https://arxiv.org/abs/2512.18261 (Survey of 60 security practitioners across 27 countries.)