Kiln » KilnSupportScripts Powershell Scripts to help monitor a Kiln environment. Contact Fog Creek support before using. Read More
Clone URL:  
Pushed to one repository · View In Graph Contained in tip

add script to churn through the Kiln Queue

Changeset 51cec4d6604b…

Parent 0ada79e77615…

by Profile picture of User 476Ben McCormack <benm@fogcreek.com>

Changes to one file · Browse files at 51cec4d6604b Showing diff from parent 0ada79e77615 Diff from another changeset...

Change 1 of 1 Show Entire File misc/​churnQueue.ps1 Stacked
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
 
@@ -0,0 +1,77 @@
+### This function will churn through the Kiln queue manually. The QueueService +### should be turned off before running this script. Run this script like so: +### +### > . .\churnQueue.ps1 +### > churn-queue +### +### Note that this script will automatically restart the Kiln Storage Service +### if it encounters an exception. This is mainly to deal with a socket +### limitation on Win 2003, but it may help in Win 2008 environments as well. + +function get-queueLength(){ + try { + $s = (New-Object net.webclient).DownloadString('http://localhost:56785/stats.json') + } + catch { + return "queue length unavailable" + } + $queueLength = $s -split (',') | foreach{if ($_ | select-string "queueLength" -quiet){ ($_ -split ":")[1]}} + return $queueLength +} +function churn-queue(){ + pushd + cd "C:\Program Files\Kiln\queue" + # We want to stop the Kiln Queuing Service because we're going to be running it + # manually. + write "---------------- Stopping Queuing Service ----------------" + sc.exe stop "Kiln Queuing Service" + $continue = $true + $restartKss = $false + # We're going to loop until we're out of work for the queue service. + while ($continue){ + if ($restartKss) { + # With the QueueService hammering Redis, this can cause Redis to fail at the + # network level. This causes the Queuing Service to fail. Restarting the + # Kiln Storage Service restarts Redis so that the Queuing Service can continue. + $restartKss = $false + write "---------------- Restart Kiln Storage Service ----------------" + sc.exe stop "KilnStorageService" + sc.exe start "KilnStorageService" + } + $getQueueLength = $true + # The following code is going to process the Queue Service manually, printing the + # output. Powershell is then piping the output as a stream, which is tested for + # various conditions to determine if the process is working. + write "---------------- Churn Through Kiln Queue ----------------" + . .\QueueService.exe /verbose /noservice | out-string -stream | foreach{ + if ($getQueueLength) { + # We'll use get-queueLength (defined above) to query the queue service + # stats to see how many items we have left in the queue. This number + # should go down over time. We'll query it once each time we start the + # Queue Service. + $getQueueLength = $false + $queueLength = get-queueLength + write "---------------- Get Kiln Queue Length ----------------" + write "KILN QUEUE LENGTH: $queueLength" + } + write $_ + if($_ | select-string "no work" -quiet){ + # If there are no items left for the Queue, we'll see a message that says "no work", + # at which point we'll want to exit the entire loop. + write "---------------- No more Work in the Queue ----------------" + $continue = $false + continue + } + if($_ | select-string "Exception" -quiet){ + # If we see the word "Exception", this almost certainly means that Redis has crashed. + # We're going set a flag to restart the Kiln Storage Service, then continue to + # restart the current loop. The Storage Service will get restarted at the top of + # the loop. + write "---------------- Exception While Processing the Queue ----------------" + $restartKss = $true + continue + } + } + } + popd +} \ No newline at end of file