Session given by Alex Jauch – @AJauch vSphere APIs for IO Filtering (VAIO) -> vSphere APIs for IO Filtering (VAIO) which creates an IO filtering framework allowing customer provided filters to process a VMs IO to its VMDKs. This has deep implication for providers who would have looked to write kernel mode drivers to implement things like distributed caching layers or data services.  Filters will be inside ESXi and outside the VM. No more kernel mode drivers!

Basically, VAIO makes calls more efficient “hard coded up-calls with ring buffer” from user mode, increasing performance (1 microsecond latency) without making the entire kernel unstable. This allows for full third party data services without full blown kernel mode access. Why? Enable Ecosystem; Provide customer choice; OOB release schedule; ESXi stability VAIO gives unfettered access to the IO Stream of a VM. Nothing sits between the IO of the VM and the filter framework. Filters are in user space, so full POSIX framework, C library, etc.

VAIO v1 Use Cases Cache -> onboard SSD access, Read/Write Cache; distributed cache management Replication -> Sync access to VM IO event queue; Full IP sockets interface for replication, dynamic load/unload of filters

What does VAIO give you?
-Register for VMDK IO stream and you get everything from that VMDK vSCSI IO event queue
-Tight integration of Vsphere  (integrated mgmt VASA and sPBM)
-Can alter IOs before they go to disk
-Automated Deployment (deployed as VIBs) -Flexible -> Full C Libarary Support (extensible va vC extentions)

Conceptual Design   20140825_154826  
  1. VM issues normal IO
  2. Register VM to AI framework redirects I/O to filter
  3. after processing the filter returns processing to VAI framework
  4. I/O continues down I/O pipeline
  5. I/O result is returned to storage subsystem
  6. I/O result is redirected to the filter
  7. After processing the result is returned to VAIO framework
  8. When all filters are done, VAIO returns the io result to the guest via existing vscsi infrastructure
Filter can issue IO if you want to impersonate the VM. IOs appear to come directly from the VM as the vSCSI request converted to IOCTL then passed to VAIO. If you are using direct path which bypasses the vSCSI stack, VAIO will not see the traffic (applies to RDMs as well). VAIO can work with anything that shows up as IO device. Filter is not visible to guest, it knows nothing about it.

Design Constraints
  • Filters run in user space, no vmkernel
  • filter objective lives above the system storage stack and below the VM
  • filter is deployed per VMDK, cannot be deployed at the DS level and has no notion of data stores
  • there will be an eventing mechanism to let you know when a snapshot or vmotoin occurs, but we will not provide any cache coherency or other data replication mechanisms
  • filters will only be in C
  • will need 32 bit and 64 bit versions (ESXi is a 32bit OS with 64bit address space.)
  • one instance per vmx, must be re-entrant as we will be multi-thread in our callbacks
Demo showing a SPBM integration. Included filter that allowed inserting cache engine backed by a SSD without having to provision a datastore on that SSD. Allowed using it as a ‘native’ cache 🙂

Q4 2014 release for this feature. “RC” quality build with ESXi and vSphere. Requires kernel level changes in ESXi and vSphere. Not backwards compatible and will work with ESXi 6 only.  Filters will need to be signed by VMware before they can be implemented. VAIO DK GA early in 2015 (after GA of ESXi 6 GA).

Other Misc Notes: Shadow VM (user world) is used for offline access or when you have processes that need to communicate across worlds (like distributed cache or metadata mapping). IO Filter (VM) and storage profile (datastore) are separate entities. Filter must be running in the cluster you put the VMs. You could split writes as well (for replicating or protecting data to two different VMDKs).

3 thoughts on “TEX1492 – IO Filters: Adding Data Services to ESXi

  1. Pingback: VMworld 2014: 2 Days In—Much to Think About | SolidFire | Blog

  2. Pingback: VMworld 2014 Scaled-Out Recap | @keithnorbie virtual ideas

  3. Pingback: VMware Previews VAIO - vSphere APIs for I/O Filtering | Architecting IT Blog

Leave a Reply

Your email address will not be published. Required fields are marked *