You can see how write-caching is handled for a particular volume by right-clicking on the icon for the drive in the "Disk drives" subtree of the Device Manager and selecting the Policies tabs.
The options typically are "Optimize for quick removal" (everything is written to the drive immediately) and "Optimize for performance" (writes are cached).
The first option lets you quickly disconnect drives -- for instance, hot-pluggable USB "pen" drives -- without first disconnecting them via the Safely Remove Hardware icon in the system tray.
If all the options are greyed out, this means write-caching is probably handled at the discretion of the device driver.
Make sure both drives are "Optimize for quick removal"