4、TwinVisor：HardwareisolatedConfidentialVirtualMachinesforARM.pdf

1、TwinVisor:Hardware-isolated Confidential Virtual Machines for ARM糜泽羽上海交通大学并行与分布式系统研究所(IPADS)https:/ computing grows rapidlyTenants are entrusting data with cloudData breach is a daily occurrenceE.g.,Azures cloud database breached in 20201Data Security in Cloud is Important1https:/msrc- 2025,50%of la

2、rge organizations will adopt privacy-enhancing computation for processing data in untrusted environments and multiparty data analytics use cases.”-Gartner上海交通大学并行与分布式系统研究所（IPADSSJTU）Hardware-isolated environments Data is only visible to authorized codeConfidential Computing is Emerging上海交通大学并行与分布式系统

3、研究所（IPADSSJTU）Hardware-isolated environments Data is only visible to authorized codeSecure enclaves are deployed in cloud Application-level confidential computing E.g.,Azure SQL database1based on Intel SGXConfidential Computing is EmergingContainerKernelCodeDataHardwareAPPEnclave1https:/ of Confiden

4、tial Hardware5IntelAMDARMAnnounce SGX2014SGX is available(6-gen core)20152016AnnounceSGXv22019SGXv2 isavailable2020AnnounceTDX2021TDXScalable SGXSEV&SME20162017SEV-ES2020AnnounceSEV-SNP2021SEV-SNP isavailableTrustZone20042017Secure-EL22021ARM CCA上海交通大学并行与分布式系统研究所（IPADSSJTU）OS-level confidential comp

5、utingProtect both kernel and user modesConfidential VM has gained traction Good compatibility with IaaS Minimal intrusiveness to workload Clear security boundaryConfidential VM in CloudGoogle CloudConfidential VMMicrosoftAzure Confidential ComputingIBM Hybrid Cloud上海交通大学并行与分布式系统研究所（IPADSSJTU）Existin

6、g Confidential VM SolutionsSpecialized hardware+Shared hypervisorHypervisorDRAMVM 0Mem Controller0VM 11VMVMHypervisorDRAMVM 0TDX Module0VM 11VMVMHypervisorDRAMVM 0UltravisorVM 1VMVMAMD SEV(x86):-Dedicated secure processorIntel TDX(x86):-Hardware shim layer-Microcode:TDX moduleIBM PEF(Power):-Additio

7、nal privilege level-Software:Ultravisor上海交通大学并行与分布式系统研究所（IPADSSJTU）Intel Trusted Domain Extensions(TDX)Shield VMs(TD)from any other non-TD software Virtual Machine Extensions(VMX)Multi-key,total memory-encryption(MKTME)technology CPU-attested,software moduleTDX protects the TDs with physical attacks

8、 Cold-boot attacks Memory relocating,splicing,and aliasing attacks Not defend against replay attack8HypervisorDRAMVM 0TDX Module0VM 11VMVM上海交通大学并行与分布式系统研究所（IPADSSJTU）Why ARM becomes popular in cloud?Rich ecosystem Excellent price-performance ratioDo ARM users care about the security of their data?Of

9、 course they do!Confidential VMs on ARM:no available solution yetWhen and how ARM servers will support confidential VMs?ARM Rises in Cloud Computing上海交通大学并行与分布式系统研究所（IPADSSJTU）ARM Confidential Compute Architecture10ARMv9s solution to the future needs of securityExpand to four world states Non-secure

10、,Realm,secure,rootSecure monitorVMMOSAPPAPPTEEOSSEL2RMMOSAPPRealmNon-secureSecureRootNon-secureMemRealmMemSecureMemRootMemNon-secureRealmSecureRoot上海交通大学并行与分布式系统研究所（IPADSSJTU）CCA Software Architecture11Secure MonitorIsolationRealm mgt.AttestationCryptoRMMAppServiceRealmOSRealmVMHypervisorVMAppOSSecu

11、re PlatformManagerTATEEOSTATEEOSRealmNon-secureSecure上海交通大学并行与分布式系统研究所（IPADSSJTU）Twinvisor:ARM Secure VMbased on TrustZone(SOSP 2021)12上海交通大学并行与分布式系统研究所（IPADSSJTU）ARM TrustZone&Secure-EL2(S-EL2)extensionTrustZone:isolated from the normal world,widely used on mobile platformsS-EL2 enables hardware vi

12、rtualization in TrustZone since ARMv8.4 S-EL2 mirrors N-EL2ARM TrustZone&S-EL2Region-based memory isolation-Up to 8 regions(TZC-400)-Configured by EL3&S-EL2FirmwareHypervisorAPPsKernelS-EL0S-EL1S-EL2HypervisorAPPsKernelNormal WorldWith S-EL2N-EL0N-EL1N-EL2EL3SMCSMCDRAMTATEE-KernelSecure WorldWithout

13、 S-EL2TA上海交通大学并行与分布式系统研究所（IPADSSJTU）ARM TrustZone&Secure-EL2(S-EL2)extensionTrustZone:isolated from the normal world,widely used on mobile platformsS-EL2 enables hardware virtualization in TrustZone since ARMv8.4 S-EL2 mirrors N-EL2ARM TrustZone&S-EL2Region-based memory isolation-Up to 8 regions(TZC

14、-400)-Configured by EL3&S-EL2FirmwareHypervisorAPPsKernelS-EL0S-EL1S-EL2HypervisorAPPsKernelNormal WorldWith S-EL2N-EL0N-EL1N-EL2EL3SMCSMCDRAMTATEE-KernelSecure WorldWithout S-EL2TAIs it possible to retrofit TrustZonewith new software designs?上海交通大学并行与分布式系统研究所（IPADSSJTU）TrustZone-style:Another Dedic

15、ated HypervisorA straightforward design:A dedicated full-fledged hypervisor for secure VMs(S-VMs)in TrustZoneTCB:the hypervisor in S-EL2上海交通大学并行与分布式系统研究所（IPADSSJTU）A straightforward design:A dedicated full-fledged hypervisor for secure VMs(S-VMs)in TrustZoneTCB:the hypervisor in S-EL2Lessons from co

16、mmercial hypervisors and TEE-Kernels Born:small TCB,high-security guarantee Evolved:bloated TCB,numerous vulnerabilities&attack surfacesRepresentative KVM CVEs in recent five yearsRepresentative TEE system CVEs上海交通大学并行与分布式系统研究所（IPADSSJTU）TrustZone-style:Another Dedicated HypervisorOur Design:TwinVis

17、orKey observationMature hypervisors already exist in the normal worldDisentangle the management from the protectionProtectionS-VMS-EL0&1S-EL2Normal WorldN-EL0&1N-EL2EL3Secure WorldUntrustedTrustedN-VMN-VMS-VMS-visorN-visorResourceManagementFirmware上海交通大学并行与分布式系统研究所（IPADSSJTU）Our Design:TwinVisorKey

18、observationMature hypervisors already exist in the normal worldDisentangle the management from the protectionPhysical attacks,Side-channel,DoSUntrusted software-Device vendors providehardware-backed attestation-S-VMs protect their I/O dataProtectionS-VMS-EL0&1S-EL2Normal WorldN-EL0&1N-EL2EL3Secure W

19、orldUntrustedTrustedN-VMN-VMS-VMS-visorN-visorResourceManagementFirmware上海交通大学并行与分布式系统研究所（IPADSSJTU）Design Goals1.SecurityProtecting S-VMs from untrusted softwareKeeping the TCB small2.EfficiencyComparable performance and scalability to Vanilla3.Minimal modificationsMinor modifications to existing s

20、oftware上海交通大学并行与分布式系统研究所（IPADSSJTU）Challenges of Existing TrustZone1.Independent privilege modelsNo transparent trap-and-emulateIdealRealityS-visorN-visorIndependentPrivilegeS-visorN-visorHighPrivilegeLowPrivilege上海交通大学并行与分布式系统研究所（IPADSSJTU）Challenges of Existing TrustZone1.Independent privilege mod

21、elsNo transparent trap-and-emulate2.Static resource partitionsInsufficient resource/Low utilizationDRAMRegionA limited number ofcontiguous regionsDRAMUnlimited fine-grainedsecure memoryIdealRealityIdealRealityS-visorN-visorIndependentPrivilegeS-visorN-visorHighPrivilegeLowPrivilege上海交通大学并行与分布式系统研究所（

22、IPADSSJTU）Challenges of Existing TrustZone1.Independent privilege modelsNo transparent trap-and-emulate2.Static resource partitionsInsufficient resource/Low utilization3.Slow world switchesMuch runtime overhead for S-VMsIdealRealityS-visorN-visorIndependentPrivilegeS-visorN-visorHighPrivilegeLowPriv

23、ilegeDRAMRegionA limited number ofcontiguous regionsDRAMUnlimited fine-grainedsecure memoryN-visorInfrequent TA callsTEE OSS-visorFrequent VM exitsN-visorS-VMS-visorExpectedRealityIdealReality上海交通大学并行与分布式系统研究所（IPADSSJTU）Problems of dynamic secure memory by the S-visorDiscrete secure memory pages vs.

24、Limited contiguous secure memory regionsN-visors unaware of security changesAdjust Resource DynamicallyNo more regionSecureSecureLow utilizationS-VMN-visorSecureSecureNew secure page上海交通大学并行与分布式系统研究所（IPADSSJTU）ObservationOSes are usually equipped with Contiguous Memory Allocator(CMA)Split CMAKeep se

25、cure memory contiguous+Cooperative managementCooperative Resources ManagementSecureN-VMN-visorS-visor#S2PFS-VM 0S-VM 1上海交通大学并行与分布式系统研究所（IPADSSJTU）ObservationOSes are usually equipped with Contiguous Memory Allocator(CMA)Split CMAKeep secure memory contiguous+Cooperative managementCooperative Resourc

26、es ManagementSecureN-VMN-visorS-visor#S2PFS-VM 0S-VM 1S-VM 0S-VM 1MigrateN-visorS-visorN-VMSecure上海交通大学并行与分布式系统研究所（IPADSSJTU）ObservationOSes are usually equipped with Contiguous Memory Allocator(CMA)Split CMAKeep secure memory contiguous+Cooperative managementCooperative Resources ManagementSecureN-

27、VMN-visorS-visor#S2PFS-VM 0S-VM 1S-VM 0S-VM 1N-visorS-visorN-VMSecureEnlarge securememory regionS-VM 0S-VM 1MigrateN-visorS-visorN-VMSecure上海交通大学并行与分布式系统研究所（IPADSSJTU）ObservationOSes are usually equipped with Contiguous Memory Allocator(CMA)Split CMAKeep secure memory contiguous+Cooperative manageme

28、ntCooperative Resources ManagementS-VM 0N-visorS-visorN-VMSecureN-VMNormal memorynot enough上海交通大学并行与分布式系统研究所（IPADSSJTU）ObservationOSes are usually equipped with Contiguous Memory Allocator(CMA)Split CMAKeep secure memory contiguous+Cooperative managementCooperative Resources ManagementS-VM 0N-visorS

29、-visorN-VMSecureN-VMNormal memorynot enoughS-VM 0N-visorS-visorN-VMN-VMCompaction上海交通大学并行与分布式系统研究所（IPADSSJTU）ObservationOSes are usually equipped with Contiguous Memory Allocator(CMA)Split CMAKeep secure memory contiguous+Cooperative managementCooperative Resources ManagementS-VM 0N-visorS-visorN-VM

30、SecureN-VMNormal memorynot enoughS-VM 0N-visorS-visorN-VMN-VMResize securememory regionSecureS-VM 0N-visorS-visorN-VMN-VMCompaction上海交通大学并行与分布式系统研究所（IPADSSJTU）Implementation:PrototypesFunctional evaluationOfficial simulator:ARM Fixed Virtual Platform(FVP)with S-EL2 enabledPerformance evaluationReal

31、hardware:Hisilicon Kirin 990 development board(ARMv8.2)N-EL2 and Virtualization Host Extension(VHE)Work similarly to the S-EL2 enabled hardware上海交通大学并行与分布式系统研究所（IPADSSJTU）Implementation:PrototypesFunctional evaluationOfficial simulator:ARM Fixed Virtual Platform(FVP)with S-EL2 enabledPerformance eva

32、luationReal hardware:Hisilicon Kirin 990 development board(ARMv8.2)N-EL2 and Virtualization Host Extension(VHE)Work similarly to the S-EL2 enabled hardwareThe code size of the prototype system of TwinVisor上海交通大学并行与分布式系统研究所（IPADSSJTU）Performance EvaluationDevelopment boardHisilicon Kirin 990(ARMv8.2)

33、+8GB RAM+256GB ROM Only enable 4 Cortex-A55 cores(1.95 GHz)to avoid performance instabilityNetwork setupClient:Ubuntu-18.04 VM+6-core Intel i7-8700 CPU+32GB RAMTethering the board and the Ubuntu VM through USB Type-COther configurationFirmware in EL3:Trusted Firmware-A(TF-A)v1.5N-visor:Linux kernel

34、4.14+QEMU v4.2.0N-VMs&S-VMs:Linux kernel 4.15+8GB disk image上海交通大学并行与分布式系统研究所（IPADSSJTU）1-vCPU4-vCPU8-vCPUReal-world Application PerformanceNormalized performance of an S-VM compared with VanillaMaximum average overhead 5%Small overhead:world switches occupy merely a small proportion of applications

35、 effective execution time*512MB memory and different vCPUs(8-vCPU CPU oversubscription)ConclusionEnable hardware-isolated confidential VM on ARM platformsRetrofit existing TrustZone with software designsDecouple protection from managementH-trap,split CMA,fast switchAchieve comparable performance to VanillaLess than 5%overhead for all applications on SMP VMsProvide design references for future systemsE.g.,dual-hypervisor systems atop ARM CCAThanks!Open-source prototype:https:/