Essential Eight Penetration Testing: What Maturity Level 2 and 3 Actually Require

The Australian Signals Directorate Essential Eight maturity model is Australia's most widely referenced baseline for cyber security controls. At Maturity Level 2, organisations are expected to have tested the effectiveness of their controls. At Maturity Level 3, that expectation extends to adversarial simulation against the most sophisticated techniques. Most organisations conducting Essential Eight assessments treat the penetration testing requirement as a compliance activity rather than a genuine test of resilience. The result is a maturity level that looks correct on paper and fails under real conditions.

We have conducted Essential Eight assessments across a range of Australian organisations in the government, financial services, and critical infrastructure sectors. The gap between what organisations believe their maturity level reflects and what their controls actually deliver is significant. Understanding what the penetration testing requirement means at each level is the starting point for closing that gap.

What Maturity Level 2 Requires from Penetration Testing

At Maturity Level 2, the ACSC guidance requires that organisations test the effectiveness of their implemented controls against adversary techniques consistent with commodity threats. This is not a vulnerability scan. A vulnerability scanner identifies known weaknesses in software versions and configurations. It cannot test whether your application control policy is actually enforced, whether your user application hardening prevents a payload from executing, or whether your restricted administrative privileges policy holds against a lateral movement scenario.

Penetration testing at ML2 should cover the controls relevant to your environment. For an organisation with significant internet-facing services, this means testing application control bypass techniques, macro execution paths, and whether email filtering controls can be circumvented by techniques in common use. Evidence that satisfies an assessor at ML2 is a report from a qualified tester that demonstrates specific techniques were attempted against the controls and documents what was and was not effective.

What Maturity Level 3 Adds

Maturity Level 3 raises the bar considerably. The ACSC guidance at ML3 expects organisations to test against adversary techniques used by sophisticated, targeted threat actors. This moves beyond commodity malware delivery into areas including:

• Credential harvesting and pass-the-hash or pass-the-ticket attacks against privileged accounts
• Living-off-the-land techniques that use legitimate system tools rather than identifiable malware
• Bypass of endpoint detection controls using in-memory execution or trusted process injection
• Persistence mechanisms that survive credential rotation and reimaging
• Lateral movement within a segmented network environment

Testing at ML3 is closer in nature to a red team exercise than a standard penetration test. The tester is not simply looking for vulnerabilities; they are attempting to complete an objective -- typically access to a sensitive system or data set -- using techniques that a targeted attacker would use. The evidence an assessor looks for at ML3 is documentation that these techniques were actively tested and that the organisation's detection and response controls identified and contained the activity.

Evidence That Satisfies an Assessor

ACSC assessors and independent auditors conducting Essential Eight assessments look for specific evidence when evaluating whether the penetration testing requirement has been met. A vendor invoice and a tick in a spreadsheet do not satisfy the requirement. What they look for includes:

• A scoped report from a qualified tester that identifies the controls tested, the techniques used, and the outcomes
• Evidence that findings from previous tests were remediated or formally accepted
• Records showing the testing occurred within a timeframe consistent with the organisation's change rate -- annual testing is the typical minimum, but organisations with significant changes to their environment should test more frequently
• At ML3, evidence of detection: logs, alerts, or incident records showing that security operations identified the simulated activity

The testing should be conducted by testers who hold relevant qualifications and who operate under a documented scope agreement. For Australian government agencies, this typically means testers who hold an active security clearance and whose firm is assessed as capable of conducting the required techniques safely in a production-adjacent environment.

Common Gaps We See in Practice

The most frequent gap we observe is organisations that have completed an annual external penetration test and used that as their evidence for all eight controls. An external penetration test covers network and application attack surface. It does not test application control enforcement, it does not test administrative privilege restrictions in a meaningful way, and it does not test patch management effectiveness unless the scope specifically includes those controls.

A second common gap is re-testing. Organisations test, receive findings, partially remediate, and then treat the original test as their evidence of control effectiveness. The evidence trail needs to show that the specific findings were remediated and that the fix held under re-testing. A finding that was identified twelve months ago and is still open is not evidence of ML2 compliance in the control it relates to.

To discuss Essential Eight penetration testing requirements for your organisation, contact Cyberlinx at info@cyberlinx.com.au.