Kiln » KilnSupportScripts Powershell Scripts to help monitor a Kiln environment. Contact Fog Creek support before using. Read More
Clone URL:  
Pushed to one repository · View In Graph Contained in tip

Merge

Changeset a141820a4935

Parents 13315a17166d

Parents 8087f38d2aea

by Profile picture of User 476Ben McCormack <benm@fogcreek.com>

Changes to 4 files · Browse files at a141820a4935 Showing diff from parent 13315a17166d 8087f38d2aea Diff from another changeset...

Change 1 of 2 Show Entire File README.TXT Stacked
renamed from readMe.txt
 
 
 
 
 
1
2
3
 
10
11
12
13
14
15
 
 
 
16
17
18
19
20
 
21
22
 
 
 
23
24
 
25
26
 
 
 
 
 
 
27
28
29
30
 
 
 
31
32
 
 
 
 
 
33
 
 
 
 
 
 
34
 
 
 
 
 
 
 
 
35
36
37
38
39
40
 
41
42
 
 
43
44
45
46
47
48
49
50
 
51
52
53
54
55
 
 
56
57
 
 
 
 
58
59
 
 
 
 
 
60
61
 
 
62
63
 
 
 
 
 
 
 
 
 
64
65
 
 
 
 
 
 
 
 
 
 
 
66
67
 
 
 
68
69
 
70
71
72
73
 
 
 
 
 
 
1
2
3
4
5
6
7
 
14
15
16
 
 
 
17
18
19
20
21
22
23
 
24
25
 
26
27
28
29
 
30
31
 
32
33
34
35
36
37
38
39
40
 
41
42
43
44
 
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
 
 
 
 
 
67
68
 
69
70
71
 
 
 
 
 
 
 
72
73
74
75
76
77
78
79
80
 
81
82
83
84
85
 
86
87
88
89
90
91
 
92
93
94
 
95
96
97
98
99
100
101
102
103
104
 
105
106
107
108
109
110
111
112
113
114
115
116
 
117
118
119
120
 
121
122
 
 
 
 
123
124
125
 
@@ -1,3 +1,7 @@
+ATTENTION: THESE SCRIPTS HAVE NOT BEEN TESTED IN A PRODUCTION ENVIRONMENT!!! +IF YOU FEEL YOU ARE HAVING ISSUES WITH YOUR KILN INSTALLATION, PLEASE CONTACT +FOG CREEK SUPPORT AT CUSTOMER-SERVICE@FOGCREEK.COM. +  This repository contains a series of Powershell scripts that can be used  to check up on Kiln, log environment data, restart the backend if necessary,  and even send email for alerts. @@ -10,64 +14,112 @@
    - checkBackend.ps1 Contains the Powershell functions to check Kiln   - test-checkBackend.ps1 Use this file to test functions - - run-checkBackend.example.ps1 This is an example script that uses the functions - in checkBackend.ps1. - - tools.ps1 Helpers for parsing the log file + - run-checkBackend.example.ps1 This is an example script that uses the + functions in checkBackend.ps1. + - tools.ps1 Helpers for parsing the log file      Here's a more detailed explanation for some of the files   -checkBackend.ps1 +# checkBackend.ps1   -This is the main file that contains the methods used by the other scripts. Nothing is actually executed when you simply run the file using . .\checkBackend.ps1 , but instead, the functions are loaded into memory. +This is the main file that contains the methods used by the other scripts. +Nothing is actually executed when you simply run the file using +`. .\checkBackend.ps1`, but instead, the functions are loaded into memory.   -test-checkBackend.ps1 +# test-checkBackend.ps1   -You can use this file to make sure your variables are set up correctly. You'll need to update the first line to correctly point to the checkBackend.ps1. Test-KilnVersion requires no parameters, so there's nothing we need to do here. However, Test-KilnDiff requires actual repository information, so we're likely going to want to go to the database to get this data. (You could use an existing /repo/{GUID}/diff/... URL that you've seen earlier, but this might be for a very large diff, and if possible, we'd like to use a much smaller diff). +You can use this file to make sure your variables are set up correctly. You'll +need to update the first line to correctly point to the checkBackend.ps1. +Test-KilnVersion requires no parameters, so there's nothing we need to do here. +However, Test-KilnDiff, which gets a diff from the Kiln Storage Service, +requires actual repository information, so we're likely going to want to go to +the database to get this data.    $sRepoGUID, $sChangeset1, and $sChangeset2   -What we're looking for is a relatively small diff (a few lines) in a repository that we don't expect to be deleted. This will help us ensure that our Test-KilnDiff method runs reliably without consuming too many resources. +What we're looking for is a relatively small diff (a few lines) in a repository +that we don't expect to be deleted. This will help us ensure that our +Test-KilnDiff method runs reliably without consuming too many resources.   -You can use the Kiln UI to find a changeset with a diff that appears relatively small. Once you find a changeset, view it in the Kiln UI, and not the partial changesetID in the URL, e.g. .../History/f181ad063566. Once you've noted the changesetID, run the following query in the Kiln database, updating the Where clause accordingly: +You can use the Kiln UI to find a changeset with a diff that appears relatively +small. Once you find a changeset, view it in the Kiln UI, and note the partial +changesetID in the URL, e.g. .../History/f181ad063566. Once you've noted the +changesetID, run the following query in the Kiln database, updating the Where +clause accordingly:   + Select Repo.sName, Repo.UUid, Changeset.sID, Changeset.sParentID1 + From Changeset + Inner Join ChangesetPush + On Changeset.ixChangeset = ChangesetPush.ixChangeset + Inner Join Repo On ChangesetPush.ixRepo = Repo.ixRepo + Where Left(sID,12) = 'f181ad063566'   +This is going to give you the UUid, sID, and sParentID1, which you'll map to +$sRepoGUID, $sChangeset1, and $sChangeset2, accordingly. + +Testing values in test-checkBackend.ps1: + +You can go ahead and update $logFile as well. It's not used in the test, but +if you copy variables over to run-checkBackend.example.ps1, they might as well +be the same.   -Select Repo.sName, Repo.UUid, Changeset.sID, Changeset.sParentID1 -From Changeset -Inner Join ChangesetPush On Changeset.ixChangeset = ChangesetPush.ixChangeset -Inner Join Repo On ChangesetPush.ixRepo = Repo.ixRepo -Where Left(sID,12) = 'f181ad063566' +$kilnRepositories is used to find miniredis.db, so update it accordingly.   -This is going to give you the UUid, sID, and sParentID1, which you'll map to $sRepoGUID, $sChangeset1, and $sChangeset2, accordingly. +Once all the values have been updated, navigate to the directory from the +powershell command line and run:   -Testing values - -You can go ahead and update $logFile as well. It's not used, in the test, but if you copy variables over to run-checkBackend.ps1, they might as well be the same. I don't believe the file needs to exist yet, but the directory probably needs to exist. - -Once all the values have been updated, navigate to the directory from the powershell command line and run: - -. .\test-checkBackend.ps1 + . .\test-checkBackend.ps1    You should see:    True  True +The size of miniredis.db is: 123 +123   -The first True indicates that it was able to ping the backend for version. The second True indicates that it was successfully able to run the Diff. If you get a false back for both, then either the backend is completely down or something is wrong with our script. +The first True indicates that it was able to ping the backend for version. +The second True indicates that it was successfully able to run the Diff. If +you get a false back for both, then either the backend is completely down or +something is wrong with our script.   -If you get a False back for the second value, it's likely an issue with the repo and changeset variables that you have. +If you get a False back for the second value, it's likely an issue with the +repo and changeset variables that you have. + +If you don't see a size for miniredis.db, check your $kilnRepositories +variable.   -Once you get two Trues, move to the next step. +Once you get a satisfactory output with two Trues and a size for miniredis.db, +move to the next step.   -run-checkBackend.ps1 +# run-checkBackend.example.ps1 + +This script is an example of what functionality is available in +checkBackend.ps1. You may not need each bit of functionality in your +environment, so remove sections of code as appropriate. + +I explain the variables for $sRepoGUID, $sChangeset1, and $sChangeset2 in the +above section for test-checkBackend.ps1, so read the relevant info if you +haven't already.   -This file is going to use the same variables as test-checkBackend.ps1, so go ahead and copy over the first 5 lines from test- and replace the first 5 lines in run-. +This file is going to use the same variables as test-checkBackend.ps1, so go +ahead and copy over the variable lines to run-checkBackend.example.ps1. + +The script begins with logging memory usage for Kiln processes and instead of +simply testing the services, this will actively log and will indeed restart +the storage service in Check-KilnBackend if either of the Test- methods fail. +That's why it's important to make sure your variables work in +test-checkBackend.ps1 before running this script. + +If you want, you can configure it to send you an email if a certain condition +is met, such as if a check of Kiln fails or if miniredis.db is too large.   -Instead of simply testing the services, this will actively log and will indeed restart the storage service in Check-KilnBackend if either of the Test- methods fail. That's why it's important to make sure your variables work in Test before running this script. +Once you've tested your variables in test- and have copied them over to run-, +you can test this script by navigating to the directory in powershell and +running:   -Once you've tested your variables in test- and have copied them over to run-, you can test this script by navigating to the directory in powershell and running: + . .\run-checkBackend.ps1   -. .\run-checkBackend.ps1 - -Once that executes, you should see that your log file has updated. If so, proceed to the next step. \ No newline at end of file
+Once that executes, you should see that your log file has updated. If it worked +successfully, you may want to have the script run every 15 minutes as a +scheduled Windows task. \ No newline at end of file
Change 1 of 1 Show Entire File checkBackend.ps1 Stacked
 
51
52
53
 
54
55
56
57
58
59
 
60
61
62
 
51
52
53
54
55
56
57
58
59
60
61
62
63
64
@@ -51,12 +51,14 @@
  }   catch {   LogOrWrite "Could not find Kiln Repositories Directory: $kilnRepositoriesDir" $logFile + return   }   try {   $miniRedis = (Resolve-Path ($kilnRepo + "\miniredis.db") -ErrorAction Stop).path   }   catch {   LogOrWrite "Could not find miniredis.db file in $kilnRepo" $logFile + return   }   $lenMiniRedis = (get-childitem -path $miniRedis).length   LogOrWrite "The size of miniredis.db is: $lenMiniRedis" $logFile
 
4
5
6
 
7
8
9
10
11
12
13
 
4
5
6
7
8
9
10
 
11
12
13
@@ -4,10 +4,10 @@
 $sChangeset2 = 'ed5f14291c3ab9d9bf24dee8e1b4761799509d15'  $logFile = "C:\code\checkKiln\checkBackend.log"  $kilnRepositories = "C:\KilnRepositories" +$maxMiniRedisSize = 3000000 #3MB  $smtpServer = "testmail.example.com"  $from = "ben@example.com"  $to = "ben@example.com" -$maxMiniRedisSize = 3000000 #3MB    Get-ProcessStats "backend" $logFile  Get-ProcessStats "memcached" $logFile
 
3
4
5
 
 
6
7
8
 
 
 
 
 
3
4
5
6
7
8
9
 
 
10
11
 
@@ -3,6 +3,9 @@
 $sChangeset1 = 'f181ad06356697b8676868fec36fbe13167081dd'  $sChangeset2 = 'ed5f14291c3ab9d9bf24dee8e1b4761799509d15'  $logFile = "C:\code\checkKiln\checkBackend.log" +$kilnRepositories = "C:\KilnRepositories" +$maxMiniRedisSize = 3000000 #3MB    Test-KilnVersion -Test-KilnDiff $sRepoGUID $sChangeset1 $sChangeset2 \ No newline at end of file
+Test-KilnDiff $sRepoGUID $sChangeset1 $sChangeset2 +Get-MiniRedisSize $kilnRepositories \ No newline at end of file