Kiln » KilnSupportScripts Powershell Scripts to help monitor a Kiln environment. Contact Fog Creek support before using. Read More
Clone URL:  
Pushed to one repository · View In Graph Contained in tip

Add sleep operations to allow the server to recover from the commands. Especially useful in recovering from the exception thrown by the QueueService and automatically restarting the script. FC2295785

Changeset d81b4e4fa716…

Parent d30937bef3d5…

by Profile picture of Developer Wiki Admin AccountDeveloper Wiki Admin Account

Changes to one file · Browse files at d81b4e4fa716 Showing diff from parent d30937bef3d5 Diff from another changeset...

Change 1 of 3 Show Changes Only misc/​churnQueue.ps1 Stacked
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
 
 
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
 
 
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
 
 
83
 ### Please contact Fog Creek support before running this script.  ###  ### This function will churn through the Kiln queue manually. The QueueService  ### should be turned off before running this script. Run this script like so:  ###  ### > . .\churnQueue.ps1  ### > churn-queue  ###  ### Note that this script will automatically restart the Kiln Storage Service  ### if it encounters an exception. This is mainly to deal with a socket  ### limitation on Win 2003, but it may help in Win 2008 environments as well.    function get-queueLength(){   try {   $s = (New-Object net.webclient).DownloadString('http://localhost:56785/stats.json')   }   catch {   return "queue length unavailable"   }   $queueLength = $s -split (',') | foreach{if ($_ | select-string "queueLength" -quiet){ ($_ -split ":")[1]}}   return $queueLength  }  function churn-queue([switch]$switchGetQueueLength){   pushd   cd "C:\Program Files\Kiln\queue"   # We want to stop the Kiln Queuing Service because we're going to be running it   # manually.   write "---------------- Stopping Queuing Service ----------------"   sc.exe stop "Kiln Queuing Service"   $continue = $true   $restartKss = $false   # We're going to loop until we're out of work for the queue service.   while ($continue){   if ($restartKss) {   # With the QueueService hammering Redis, this can cause Redis to fail at the   # network level. This causes the Queuing Service to fail. Restarting the   # Kiln Storage Service restarts Redis so that the Queuing Service can continue.   $restartKss = $false   write "---------------- Restart Kiln Storage Service ----------------"   sc.exe stop "KilnStorageService" + # give the services time to clean up + sleep 5   sc.exe start "KilnStorageService"   }   $getQueueLength = $switchGetQueueLength   # The following code is going to process the Queue Service manually, printing the   # output. Powershell is then piping the output as a stream, which is tested for   # various conditions to determine if the process is working.   write "---------------- Churn Through Kiln Queue ----------------"   . .\QueueService.exe /verbose /noservice | out-string -stream | foreach{   if ($getQueueLength) {   # We'll use get-queueLength (defined above) to query the queue service   # stats to see how many items we have left in the queue. This number   # should go down over time. We'll query it once each time we start the   # Queue Service.   $getQueueLength = $false   $queueLength = get-queueLength + # give the queue some time to process + sleep 3   write "---------------- Get Kiln Queue Length ----------------"   write "KILN QUEUE LENGTH: $queueLength"   }   write $_   if($_ | select-string "no work" -quiet){   # If there are no items left for the Queue, we'll see a message that says "no work",   # at which point we'll want to exit the entire loop.   write "---------------- No more Work in the Queue ----------------"   $continue = $false   continue   }   if($_ | select-string "Exception" -quiet){   # If we see the word "Exception", this almost certainly means that Redis has crashed.   # We're going set a flag to restart the Kiln Storage Service, then continue to   # restart the current loop. The Storage Service will get restarted at the top of   # the loop.   write "---------------- Exception While Processing the Queue ----------------"   $restartKss = $true   continue   }   }   }   popd -} \ No newline at end of file
+}