Event Tracing for Windows (ETW) &.ETL traces

Link to previous blog : EventLogs

There are situations when we might need trace logs to identify an issue, Windows has these providers to trace a particular component activity. There are several ways to take traces for any particular Provider for instance this can be achieved from Data Collector Sets in Performance Monitor.
I think the easier way to do this would be enable Analytical & debug logging, In this example I am trying to show providers for /hyper-v, /Failover-cluster, & /smb, I hope this will give some idea, however if we want to enable logman trace any provider, the problem is this trace stops after a host reboot hence would not be of much help.
There are many providers available however in this section, I just wanted to show providers which may come handy in troubleshooting a specific component {I have attached list}. I tried to gather some these providers details from a 2016 server as well & there is an improvement.

Providers

logman query providers | findstr Microsoft-VirtualMachineManager

    Microsoft-VirtualMachineManager-AzureSiteRecoveryProvider {9F1615C6-0A14-4372-BEEE-140272877684}
    Microsoft-VirtualMachineManager-Debug {43526B7E-9EE3-41A7-B023-D586F355C00B}
    Microsoft-VirtualMachineManager-DHCPServer {076A89DA-2EA1-4EE2-9148-1B77D7F3AFC1}
    Microsoft-VirtualMachineManager-Perf-Console {38AFC8DC-EE36-4C87-81F6-F5292FA615D8}
    Microsoft-VirtualMachineManager-Perf-Engine {4FDB9D50-046D-44BD-A014-6B68DA1F6944}
    Microsoft-VirtualMachineManager-Reliability {8B50A529-416D-46F9-B97A-6FEC0BD46DDE}
    Microsoft-VirtualMachineManager-Server {9F1615C6-0A14-4372-BEEE-140272877685}

logman query providers | findstr Microsoft-Windows-Hyper-V

    Microsoft-Windows-Hyper-V-Config {02F3A5E3-E742-4720-85A5-F64C4184E511}
    Microsoft-Windows-Hyper-V-EmulatedNic {09242393-1349-4F4D-9FD7-59CC79F553CE}
    Microsoft-Windows-Hyper-V-High-Availability {64E92ABC-910C-4770-BD9C-C3C54699B8F9}
    Microsoft-Windows-Hyper-V-Hypervisor {52FC89F8-995E-434C-A91E-199986449890}
    Microsoft-Windows-Hyper-V-Integration {2B74A015-3873-4C56-9928-EA80C58B2787}
    Microsoft-Windows-Hyper-V-Integration-GuestInterface {5B924BE5-409D-4C4D-91AC-BB844D7C8D3E}
    Microsoft-Windows-Hyper-V-Integration-KvpExchange {82D60869-5ADA-4D49-B76A-309B09666584}
    Microsoft-Windows-Hyper-V-Integration-RDV {FDFF33EC-70AA-46D3-BA65-7210009FA2A7}
    Microsoft-Windows-Hyper-V-Integration-Shutdown {BC714241-8EDC-4CE3-8714-AA0B51F98FDF}
    Microsoft-Windows-Hyper-V-Integration-TimeSync {F152DC14-A3A0-4258-BECE-69A3EE4C2DE8}
    Microsoft-Windows-Hyper-V-Integration-VSS {67E605EE-A4D8-4C46-AE50-893F31E13963}
    Microsoft-Windows-Hyper-V-Netvsc {152FBE4B-C7AD-4F68-BADA-A4FCC1464F6C}
    Microsoft-Windows-Hyper-V-Shared-VHDX {BB510E5F-2EB9-491A-81E4-F04654388F2B}
    Microsoft-Windows-Hyper-V-SynthFcVdev {5B621A17-3B58-4D03-94F0-314F4E9C79AE}
    Microsoft-Windows-Hyper-V-SynthNic {C29C4FB7-B60E-4FFF-9AF9-CF21F9B09A34}
    Microsoft-Windows-Hyper-V-SynthStor {EDACD782-2564-4497-ADE6-7199377850F2}
    Microsoft-Windows-Hyper-V-VID {5931D877-4860-4EE7-A95C-610A5F0D1407}
    Microsoft-Windows-Hyper-V-VMMS {6066F867-7CA1-4418-85FD-36E3F9C0600C}
    Microsoft-Windows-Hyper-V-VmSwitch {67DC0D66-3695-47C0-9642-33F76F7BD7AD}
    Microsoft-Windows-Hyper-V-Worker {51DDFA29-D5C8-4803-BE4B-2ECB715570FE}

logman query providers | findstr Microsoft-Windows-Hyper-V {addition in 2016}

    Microsoft-Windows-Hyper-V-Chipset {DE9BA731-7F33-4F44-98C9-6CAC856B9F83}
    Microsoft-Windows-Hyper-V-Compute {17103E3F-3C6E-4677-BB17-3B267EB5BE57}
    Microsoft-Windows-Hyper-V-ComputeLib {AF7FD3A7-B248-460C-A9F5-FEC39EF8468C}
    Microsoft-Windows-Hyper-V-Config {02F3A5E3-E742-4720-85A5-F64C4184E511}
    Microsoft-Windows-Hyper-V-Debug {EDED5085-79D0-4E31-9B4E-4299B78CBEEB}
    Microsoft-Windows-Hyper-V-DynMem {B1D080A6-F3A5-42F6-B6F1-B9FD86C088DA}
    Microsoft-Windows-Hyper-V-EmulatedDevices {DA5A028B-B248-4A75-B60A-024FE6457484}
    Microsoft-Windows-Hyper-V-EmulatedNic {09242393-1349-4F4D-9FD7-59CC79F553CE}
    Microsoft-Windows-Hyper-V-EmulatedStor {86E15E01-EDF1-4AC7-89CF-B19563FD6894}
    Microsoft-Windows-Hyper-V-Guest-Drivers-Dynamic-Memory {BA2FFB5C-E20A-4FB9-91B4-45F61B4B66A0}
    Microsoft-Windows-Hyper-V-Guest-Drivers-IcSvc {C18672D1-DC18-4DFD-91E4-170CF37160CF}
    Microsoft-Windows-Hyper-V-Guest-Drivers-Storage-Filter {0B9FDCCC-451C-449C-9BD8-6756FCC6091A}
    Microsoft-Windows-Hyper-V-Guest-Drivers-Vmbus {F2E2CE31-0E8A-4E46-A03B-2E0FE97E93C2}
    Microsoft-Windows-Hyper-V-High-Availability {64E92ABC-910C-4770-BD9C-C3C54699B8F9}
    Microsoft-Windows-Hyper-V-Hypervisor {52FC89F8-995E-434C-A91E-199986449890}
    Microsoft-Windows-Hyper-V-Integration {2B74A015-3873-4C56-9928-EA80C58B2787}
    Microsoft-Windows-Hyper-V-Integration-RDV {FDFF33EC-70AA-46D3-BA65-7210009FA2A7}
    Microsoft-Windows-Hyper-V-Serial {8F9DF503-1D12-49EC-BB28-F6EC42D361D4}
    Microsoft-Windows-Hyper-V-Shared-VHDX {BB510E5F-2EB9-491A-81E4-F04654388F2B}
    Microsoft-Windows-Hyper-V-StorageVSP {10B3D268-9782-49A4-AACC-A93C5482CB6F}
    Microsoft-Windows-Hyper-V-SynthFcVdev {5B621A17-3B58-4D03-94F0-314F4E9C79AE}
    Microsoft-Windows-Hyper-V-SynthNic {C29C4FB7-B60E-4FFF-9AF9-CF21F9B09A34}
    Microsoft-Windows-Hyper-V-SynthStor {EDACD782-2564-4497-ADE6-7199377850F2}
    Microsoft-Windows-Hyper-V-Tpm {13EAE551-76CA-4DDC-B974-D3A0F8D44A03}
    Microsoft-Windows-Hyper-V-UiDevices {339AAD0A-4124-4968-8147-4CBBB1F8B3D5}
    Microsoft-Windows-Hyper-V-VfpExt {9F2660EA-CFE7-428F-9850-AECA612619B0}
    Microsoft-Windows-Hyper-V-VID {5931D877-4860-4EE7-A95C-610A5F0D1407}
    Microsoft-Windows-Hyper-V-VmbusVdev {177D1599-9764-4E3A-BF9A-C86887AADDCE}
    Microsoft-Windows-Hyper-V-VMMS {6066F867-7CA1-4418-85FD-36E3F9C0600C}
    Microsoft-Windows-Hyper-V-VMSP {1CEB22B1-97FF-4703-BEB2-333EB89B522A}
    Microsoft-Windows-Hyper-V-VmSwitch {67DC0D66-3695-47C0-9642-33F76F7BD7AD}
    Microsoft-Windows-Hyper-V-VSmb {7B0EA079-E3BC-424A-B2F0-E3D8478D204B}
    Microsoft-Windows-Hyper-V-Worker {51DDFA29-D5C8-4803-BE4B-2ECB715570FE}
    Microsoft-Windows-Hyper-V-WorkerManager {8B0287F8-755D-4BC8-BD76-4CE327C4B78B}

logman query providers | findstr Microsoft-Windows-Failover

    Microsoft-Windows-FailoverClustering {BAF908EA-3421-4CA9-9B84-6689B8C6F85F}
    Microsoft-Windows-FailoverClustering-Client {A82FDA5D-745F-409C-B0FE-18AE0678A0E0}
    Microsoft-Windows-FailoverClustering-CsvFlt-Diagnostic {151D3C03-E442-4C4F-AF20-BD48FF41F793}
    Microsoft-Windows-FailoverClustering-CsvFs-Diagnostic {6A86AE90-4E9B-4186-B1D1-9CE0E02BCBC1}
    Microsoft-Windows-FailoverClustering-Manager {11B3C6B7-E06F-4191-BBB9-7099FFF55614}
    
	New "FailoverClustering" Providers in 2016

    Microsoft-Windows-FailoverClustering {BAF908EA-3421-4CA9-9B84-6689B8C6F85F}
    Microsoft-Windows-FailoverClustering-Client {A82FDA5D-745F-409C-B0FE-18AE0678A0E0}
    Microsoft-Windows-FailoverClustering-ClusBflt-Diagnostic {923BCB94-58D2-42BE-BBA9-B1315F363838}
    Microsoft-Windows-FailoverClustering-ClusDisk-Diagnostic {7FEF367F-E76C-4592-9912-E12B36A99780}
    Microsoft-Windows-FailoverClustering-Clusport-Diagnostic {29C07D0E-E5A0-4E85-A004-1F668531CE22}
    Microsoft-Windows-FailoverClustering-CsvFlt-Diagnostic {151D3C03-E442-4C4F-AF20-BD48FF41F793}
    Microsoft-Windows-FailoverClustering-CsvFs-Diagnostic {6A86AE90-4E9B-4186-B1D1-9CE0E02BCBC1}
    Microsoft-Windows-FailoverClustering-Manager {11B3C6B7-E06F-4191-BBB9-7099FFF55614}
    Microsoft-Windows-FailoverClustering-NetFt {C1FCCEB3-3F19-42A9-95B9-27B550FA1FBA}
    Microsoft-Windows-FailoverClustering-Reflector-Diagnostic {3FC9C08B-78B6-4DB3-AC62-FFFD30679757}
    Microsoft-Windows-FailoverClustering-SoftwareStorageBusTarget {0AC0708A-A44E-49EF-AA7E-FBE8CCC603A6}
    Microsoft-Windows-FailoverClustering-WMIProvider {0461BE3C-BC15-4BAD-9A9E-51F3FADFEC75}

logman query providers | findstr Microsoft-Windows-SMB

    Microsoft-Windows-SMBClient {988C59C5-0A1C-45B6-A555-0C62276E327D}
    Microsoft-Windows-SMBDirect {DB66EA65-B7BB-4CA9-8748-334CB5C32400}
    Microsoft-Windows-SMBServer {D48CE617-33A2-4BC3-A5C7-11AA4F29619E}
    Microsoft-Windows-SMBWitnessClient {32254F6C-AA33-46F0-A5E3-1CBCC74BF683}
    Microsoft-Windows-SMBWitnessService {CE704B50-B105-4BC8-A24F-1792C0401C2A}
    Microsoft-Windows-SmbWmiProvider {50B9E206-9D55-4092-92E8-F157A8235799} 

VSS Trace Provider:

logman query providers | findstr /i VSS
Microsoft-Windows-Hyper-V-Integration-VSS {67E605EE-A4D8-4C46-AE50-893F31E13963}
VSS tracing provider {9138500E-3648-4EDB-AA4C-859E9F7B7C38}

we can enable a tracing on any of of these provider {data collector set}, we can use below mentioned command {logman}to enable & start a trace , once the trace is taken, you can review the generated .etl file .

logman create trace storage -o  -p "Provider GUID" -f bincirc -max 1000
logman start storage
logman stop storage

Instead of using logman mostly I enable inbuild .etl files which can be enabled from eventvwr. We can enable Analytics logs on the providers.

Event Channels{Wevtutil}

wevtutil el | findstr /i “hyper-v failover smb”

    Microsoft-Windows-BranchCacheSMB/Analytic
    Microsoft-Windows-BranchCacheSMB/Operational
    Microsoft-Windows-FailoverClustering-Client/Diagnostic
    Microsoft-Windows-FailoverClustering-ClusBflt/Diagnostic
    Microsoft-Windows-FailoverClustering-ClusBflt/Management
    Microsoft-Windows-FailoverClustering-ClusBflt/Operational
    Microsoft-Windows-FailoverClustering-ClusRflr/Diagnostic
    Microsoft-Windows-FailoverClustering-ClusRflr/Operational
    Microsoft-Windows-FailoverClustering-Clusport/Diagnostic
    Microsoft-Windows-FailoverClustering-Clusport/Operational
    Microsoft-Windows-FailoverClustering-CsvFlt/Diagnostic
    Microsoft-Windows-FailoverClustering-CsvFs/Diagnostic
    Microsoft-Windows-FailoverClustering-CsvFs/Operational
    Microsoft-Windows-FailoverClustering-Manager/Admin
    Microsoft-Windows-FailoverClustering-Manager/Diagnostic
    Microsoft-Windows-FailoverClustering-Manager/Tracing
    Microsoft-Windows-FailoverClustering-NetFt/Diagnostic
    Microsoft-Windows-FailoverClustering-NetFt/Operational
    Microsoft-Windows-FailoverClustering-WMIProvider/Admin
    Microsoft-Windows-FailoverClustering-WMIProvider/Diagnostic
    Microsoft-Windows-FailoverClustering/Diagnostic
    Microsoft-Windows-FailoverClustering/DiagnosticVerbose
    Microsoft-Windows-FailoverClustering/Operational
    Microsoft-Windows-FailoverClustering/Performance-CSV

    Microsoft-Windows-Hyper-V-Compute-Admin
    Microsoft-Windows-Hyper-V-Compute-Analytic
    Microsoft-Windows-Hyper-V-Compute-Operational
    Microsoft-Windows-Hyper-V-Config-Admin
    Microsoft-Windows-Hyper-V-Config-Analytic
    Microsoft-Windows-Hyper-V-Config-Operational
    Microsoft-Windows-Hyper-V-Guest-Drivers/Admin
    Microsoft-Windows-Hyper-V-Guest-Drivers/Analytic
    Microsoft-Windows-Hyper-V-Guest-Drivers/Debug
    Microsoft-Windows-Hyper-V-Guest-Drivers/Diagnose
    Microsoft-Windows-Hyper-V-Guest-Drivers/Operational
    Microsoft-Windows-Hyper-V-High-Availability-Admin
    Microsoft-Windows-Hyper-V-High-Availability-Analytic
    Microsoft-Windows-Hyper-V-Hypervisor-Admin
    Microsoft-Windows-Hyper-V-Hypervisor-Analytic
    Microsoft-Windows-Hyper-V-Hypervisor-Operational
    Microsoft-Windows-Hyper-V-Shared-VHDX/Diagnostic
    Microsoft-Windows-Hyper-V-Shared-VHDX/Operational
    Microsoft-Windows-Hyper-V-Shared-VHDX/Reservation
    Microsoft-Windows-Hyper-V-StorageVSP-Admin
    Microsoft-Windows-Hyper-V-VID-Admin
    Microsoft-Windows-Hyper-V-VID-Analytic
    Microsoft-Windows-Hyper-V-VMMS-Admin
    Microsoft-Windows-Hyper-V-VMMS-Analytic
    Microsoft-Windows-Hyper-V-VMMS-Networking
    Microsoft-Windows-Hyper-V-VMMS-Operational
    Microsoft-Windows-Hyper-V-VMMS-Storage
    Microsoft-Windows-Hyper-V-VMSP-Debug
    Microsoft-Windows-Hyper-V-VfpExt-Analytic
    Microsoft-Windows-Hyper-V-VmSwitch-Diagnostic
    Microsoft-Windows-Hyper-V-VmSwitch-Operational
    Microsoft-Windows-Hyper-V-Worker-Admin
    Microsoft-Windows-Hyper-V-Worker-Analytic
    Microsoft-Windows-Hyper-V-Worker-VDev-Analytic

    Microsoft-Windows-SMBClient/Analytic
    Microsoft-Windows-SMBClient/HelperClassDiagnostic
    Microsoft-Windows-SMBClient/ObjectStateDiagnostic
    Microsoft-Windows-SMBClient/Operational
    Microsoft-Windows-SMBDirect/Admin
    Microsoft-Windows-SMBDirect/Debug
    Microsoft-Windows-SMBDirect/Netmon
    Microsoft-Windows-SMBServer/Analytic
    Microsoft-Windows-SMBServer/Audit
    Microsoft-Windows-SMBServer/Connectivity
    Microsoft-Windows-SMBServer/Diagnostic
    Microsoft-Windows-SMBServer/Operational
    Microsoft-Windows-SMBServer/Performance
    Microsoft-Windows-SMBServer/Security
    Microsoft-Windows-SMBWitnessClient/Admin
    Microsoft-Windows-SMBWitnessClient/Informational
    Microsoft-Windows-SMBWitnessServer/Admin
    Microsoft-Windows-SmbClient/Connectivity
    Microsoft-Windows-SmbClient/Diagnostic
    Microsoft-Windows-SmbClient/Security
    SmbWmiAnalytic

Command to enable a particular trace

“wevtutil sl  /e:true”
	
For Example
1. Enabling the logs 
wevtutil sl Microsoft-Windows-Hyper-V-VMMS-Analytic /e:true 
	
2. Querying logs 
wevtutil qe Microsoft-Windows-Hyper-V-VMMS-Analytic 

This is the log file path  “%SystemRoot%\System32\Winevt\Logs”

You can then use Get-WinEvent Commands to query this Analytics Event Channel.

Log Location

dir windows\System32\Winevt\Logs | Findstr /i "vmms"
-a---- 7/22/2016 12:00 PM 69632 Microsoft-Windows-Hyper-V-VMMS-Admin.evtx
-a---- 8/2/2016 11:20 AM 4096 Microsoft-Windows-Hyper-V-VMMS-Analytic.etl --> This is the ETL file which we created earlier.
-a---- 7/22/2016 11:57 AM 69632 Microsoft-Windows-Hyper-V-VMMS-Networking.evtx
 -a---- 7/26/2016 8:37 AM 69632 Microsoft-Windows-Hyper-V-VMMS-Operational.evtx
-a---- 7/22/2016 11:15 AM 69632 Microsoft-Windows-Hyper-V-VMMS-Storage.evtx

Easiest way would be to enable them via GUI (Eventvwr.exe)

  1. Enable Analytic and Debug Logsscreen-shot-2017-01-17-at-2-32-41-pm
  2. Enable Analytic and Debug log at the Eventvwr level so you can see these advanced diagnostic traces.

screen-shot-2017-01-17-at-2-34-27-pm

Then Enable the tracing and then try to diagnose the issue. I’ll update the article more incase if I find anything interesting.

We can enable Analytics logs on the providers:

Advertisements