Funktionierender Prototyp des Serious Games zur Vermittlung von Wissen zu Software-Engineering-Arbeitsmodellen.
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

win32pdhquery.py 23KB

1 year ago
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571
  1. """
  2. Performance Data Helper (PDH) Query Classes
  3. Wrapper classes for end-users and high-level access to the PDH query
  4. mechanisms. PDH is a win32-specific mechanism for accessing the
  5. performance data made available by the system. The Python for Windows
  6. PDH module does not implement the "Registry" interface, implementing
  7. the more straightforward Query-based mechanism.
  8. The basic idea of a PDH Query is an object which can query the system
  9. about the status of any number of "counters." The counters are paths
  10. to a particular piece of performance data. For instance, the path
  11. '\\Memory\\Available Bytes' describes just about exactly what it says
  12. it does, the amount of free memory on the default computer expressed
  13. in Bytes. These paths can be considerably more complex than this,
  14. but part of the point of this wrapper module is to hide that
  15. complexity from the end-user/programmer.
  16. EXAMPLE: A more complex Path
  17. '\\\\RAISTLIN\\PhysicalDisk(_Total)\\Avg. Disk Bytes/Read'
  18. Raistlin --> Computer Name
  19. PhysicalDisk --> Object Name
  20. _Total --> The particular Instance (in this case, all instances, i.e. all drives)
  21. Avg. Disk Bytes/Read --> The piece of data being monitored.
  22. EXAMPLE: Collecting Data with a Query
  23. As an example, the following code implements a logger which allows the
  24. user to choose what counters they would like to log, and logs those
  25. counters for 30 seconds, at two-second intervals.
  26. query = Query()
  27. query.addcounterbybrowsing()
  28. query.collectdatafor(30,2)
  29. The data is now stored in a list of lists as:
  30. query.curresults
  31. The counters(paths) which were used to collect the data are:
  32. query.curpaths
  33. You can use the win32pdh.ParseCounterPath(path) utility function
  34. to turn the paths into more easily read values for your task, or
  35. write the data to a file, or do whatever you want with it.
  36. OTHER NOTABLE METHODS:
  37. query.collectdatawhile(period) # start a logging thread for collecting data
  38. query.collectdatawhile_stop() # signal the logging thread to stop logging
  39. query.collectdata() # run the query only once
  40. query.addperfcounter(object, counter, machine=None) # add a standard performance counter
  41. query.addinstcounter(object, counter,machine=None,objtype = 'Process',volatile=1,format = win32pdh.PDH_FMT_LONG) # add a possibly volatile counter
  42. ### Known bugs and limitations ###
  43. Due to a problem with threading under the PythonWin interpreter, there
  44. will be no data logged if the PythonWin window is not the foreground
  45. application. Workaround: scripts using threading should be run in the
  46. python.exe interpreter.
  47. The volatile-counter handlers are possibly buggy, they haven't been
  48. tested to any extent. The wrapper Query makes it safe to pass invalid
  49. paths (a -1 will be returned, or the Query will be totally ignored,
  50. depending on the missing element), so you should be able to work around
  51. the error by including all possible paths and filtering out the -1's.
  52. There is no way I know of to stop a thread which is currently sleeping,
  53. so you have to wait until the thread in collectdatawhile is activated
  54. again. This might become a problem in situations where the collection
  55. period is multiple minutes (or hours, or whatever).
  56. Should make the win32pdh.ParseCounter function available to the Query
  57. classes as a method or something similar, so that it can be accessed
  58. by programmes that have just picked up an instance from somewhere.
  59. Should explicitly mention where QueryErrors can be raised, and create a
  60. full test set to see if there are any uncaught win32api.error's still
  61. hanging around.
  62. When using the python.exe interpreter, the addcounterbybrowsing-
  63. generated browser window is often hidden behind other windows. No known
  64. workaround other than Alt-tabing to reach the browser window.
  65. ### Other References ###
  66. The win32pdhutil module (which should be in the %pythonroot%/win32/lib
  67. directory) provides quick-and-dirty utilities for one-off access to
  68. variables from the PDH. Almost everything in that module can be done
  69. with a Query object, but it provides task-oriented functions for a
  70. number of common one-off tasks.
  71. If you can access the MS Developers Network Library, you can find
  72. information about the PDH API as MS describes it. For a background article,
  73. try:
  74. http://msdn.microsoft.com/library/en-us/dnperfmo/html/msdn_pdhlib.asp
  75. The reference guide for the PDH API was last spotted at:
  76. http://msdn.microsoft.com/library/en-us/perfmon/base/using_the_pdh_interface.asp
  77. In general the Python version of the API is just a wrapper around the
  78. Query-based version of this API (as far as I can see), so you can learn what
  79. you need to from there. From what I understand, the MSDN Online
  80. resources are available for the price of signing up for them. I can't
  81. guarantee how long that's supposed to last. (Or anything for that
  82. matter).
  83. http://premium.microsoft.com/isapi/devonly/prodinfo/msdnprod/msdnlib.idc?theURL=/msdn/library/sdkdoc/perfdata_4982.htm
  84. The eventual plan is for my (Mike Fletcher's) Starship account to include
  85. a section on NT Administration, and the Query is the first project
  86. in this plan. There should be an article describing the creation of
  87. a simple logger there, but the example above is 90% of the work of
  88. that project, so don't sweat it if you don't find anything there.
  89. (currently the account hasn't been set up).
  90. http://starship.skyport.net/crew/mcfletch/
  91. If you need to contact me immediately, (why I can't imagine), you can
  92. email me at mcfletch@golden.net, or just post your question to the
  93. Python newsgroup with a catchy subject line.
  94. news:comp.lang.python
  95. ### Other Stuff ###
  96. The Query classes are by Mike Fletcher, with the working code
  97. being corruptions of Mark Hammonds win32pdhutil module.
  98. Use at your own risk, no warranties, no guarantees, no assurances,
  99. if you use it, you accept the risk of using it, etceteras.
  100. """
  101. # Feb 12, 98 - MH added "rawaddcounter" so caller can get exception details.
  102. import _thread
  103. import copy
  104. import time
  105. import win32api
  106. import win32pdh
  107. class BaseQuery:
  108. """
  109. Provides wrapped access to the Performance Data Helper query
  110. objects, generally you should use the child class Query
  111. unless you have need of doing weird things :)
  112. This class supports two major working paradigms. In the first,
  113. you open the query, and run it as many times as you need, closing
  114. the query when you're done with it. This is suitable for static
  115. queries (ones where processes being monitored don't disappear).
  116. In the second, you allow the query to be opened each time and
  117. closed afterward. This causes the base query object to be
  118. destroyed after each call. Suitable for dynamic queries (ones
  119. which watch processes which might be closed while watching.)
  120. """
  121. def __init__(self, paths=None):
  122. """
  123. The PDH Query object is initialised with a single, optional
  124. list argument, that must be properly formatted PDH Counter
  125. paths. Generally this list will only be provided by the class
  126. when it is being unpickled (removed from storage). Normal
  127. use is to call the class with no arguments and use the various
  128. addcounter functions (particularly, for end user's, the use of
  129. addcounterbybrowsing is the most common approach) You might
  130. want to provide the list directly if you want to hard-code the
  131. elements with which your query deals (and thereby avoid the
  132. overhead of unpickling the class).
  133. """
  134. self.counters = []
  135. if paths:
  136. self.paths = paths
  137. else:
  138. self.paths = []
  139. self._base = None
  140. self.active = 0
  141. self.curpaths = []
  142. def addcounterbybrowsing(
  143. self, flags=win32pdh.PERF_DETAIL_WIZARD, windowtitle="Python Browser"
  144. ):
  145. """
  146. Adds possibly multiple paths to the paths attribute of the query,
  147. does this by calling the standard counter browsing dialogue. Within
  148. this dialogue, find the counter you want to log, and click: Add,
  149. repeat for every path you want to log, then click on close. The
  150. paths are appended to the non-volatile paths list for this class,
  151. subclasses may create a function which parses the paths and decides
  152. (via heuristics) whether to add the path to the volatile or non-volatile
  153. path list.
  154. e.g.:
  155. query.addcounter()
  156. """
  157. win32pdh.BrowseCounters(None, 0, self.paths.append, flags, windowtitle)
  158. def rawaddcounter(self, object, counter, instance=None, inum=-1, machine=None):
  159. """
  160. Adds a single counter path, without catching any exceptions.
  161. See addcounter for details.
  162. """
  163. path = win32pdh.MakeCounterPath(
  164. (machine, object, instance, None, inum, counter)
  165. )
  166. self.paths.append(path)
  167. def addcounter(self, object, counter, instance=None, inum=-1, machine=None):
  168. """
  169. Adds a single counter path to the paths attribute. Normally
  170. this will be called by a child class' speciality functions,
  171. rather than being called directly by the user. (Though it isn't
  172. hard to call manually, since almost everything is given a default)
  173. This method is only functional when the query is closed (or hasn't
  174. yet been opened). This is to prevent conflict in multi-threaded
  175. query applications).
  176. e.g.:
  177. query.addcounter('Memory','Available Bytes')
  178. """
  179. if not self.active:
  180. try:
  181. self.rawaddcounter(object, counter, instance, inum, machine)
  182. return 0
  183. except win32api.error:
  184. return -1
  185. else:
  186. return -1
  187. def open(self):
  188. """
  189. Build the base query object for this wrapper,
  190. then add all of the counters required for the query.
  191. Raise a QueryError if we can't complete the functions.
  192. If we are already open, then do nothing.
  193. """
  194. if not self.active: # to prevent having multiple open queries
  195. # curpaths are made accessible here because of the possibility of volatile paths
  196. # which may be dynamically altered by subclasses.
  197. self.curpaths = copy.copy(self.paths)
  198. try:
  199. base = win32pdh.OpenQuery()
  200. for path in self.paths:
  201. try:
  202. self.counters.append(win32pdh.AddCounter(base, path))
  203. except win32api.error: # we passed a bad path
  204. self.counters.append(0)
  205. pass
  206. self._base = base
  207. self.active = 1
  208. return 0 # open succeeded
  209. except: # if we encounter any errors, kill the Query
  210. try:
  211. self.killbase(base)
  212. except NameError: # failed in creating query
  213. pass
  214. self.active = 0
  215. self.curpaths = []
  216. raise QueryError(self)
  217. return 1 # already open
  218. def killbase(self, base=None):
  219. """
  220. ### This is not a public method
  221. Mission critical function to kill the win32pdh objects held
  222. by this object. User's should generally use the close method
  223. instead of this method, in case a sub-class has overridden
  224. close to provide some special functionality.
  225. """
  226. # Kill Pythonic references to the objects in this object's namespace
  227. self._base = None
  228. counters = self.counters
  229. self.counters = []
  230. # we don't kill the curpaths for convenience, this allows the
  231. # user to close a query and still access the last paths
  232. self.active = 0
  233. # Now call the delete functions on all of the objects
  234. try:
  235. map(win32pdh.RemoveCounter, counters)
  236. except:
  237. pass
  238. try:
  239. win32pdh.CloseQuery(base)
  240. except:
  241. pass
  242. del counters
  243. del base
  244. def close(self):
  245. """
  246. Makes certain that the underlying query object has been closed,
  247. and that all counters have been removed from it. This is
  248. important for reference counting.
  249. You should only need to call close if you have previously called
  250. open. The collectdata methods all can handle opening and
  251. closing the query. Calling close multiple times is acceptable.
  252. """
  253. try:
  254. self.killbase(self._base)
  255. except AttributeError:
  256. self.killbase()
  257. __del__ = close
  258. def collectdata(self, format=win32pdh.PDH_FMT_LONG):
  259. """
  260. Returns the formatted current values for the Query
  261. """
  262. if self._base: # we are currently open, don't change this
  263. return self.collectdataslave(format)
  264. else: # need to open and then close the _base, should be used by one-offs and elements tracking application instances
  265. self.open() # will raise QueryError if couldn't open the query
  266. temp = self.collectdataslave(format)
  267. self.close() # will always close
  268. return temp
  269. def collectdataslave(self, format=win32pdh.PDH_FMT_LONG):
  270. """
  271. ### Not a public method
  272. Called only when the Query is known to be open, runs over
  273. the whole set of counters, appending results to the temp,
  274. returns the values as a list.
  275. """
  276. try:
  277. win32pdh.CollectQueryData(self._base)
  278. temp = []
  279. for counter in self.counters:
  280. ok = 0
  281. try:
  282. if counter:
  283. temp.append(
  284. win32pdh.GetFormattedCounterValue(counter, format)[1]
  285. )
  286. ok = 1
  287. except win32api.error:
  288. pass
  289. if not ok:
  290. temp.append(-1) # a better way to signal failure???
  291. return temp
  292. except (
  293. win32api.error
  294. ): # will happen if, for instance, no counters are part of the query and we attempt to collect data for it.
  295. return [-1] * len(self.counters)
  296. # pickle functions
  297. def __getinitargs__(self):
  298. """
  299. ### Not a public method
  300. """
  301. return (self.paths,)
  302. class Query(BaseQuery):
  303. """
  304. Performance Data Helper(PDH) Query object:
  305. Provides a wrapper around the native PDH query object which
  306. allows for query reuse, query storage, and general maintenance
  307. functions (adding counter paths in various ways being the most
  308. obvious ones).
  309. """
  310. def __init__(self, *args, **namedargs):
  311. """
  312. The PDH Query object is initialised with a single, optional
  313. list argument, that must be properly formatted PDH Counter
  314. paths. Generally this list will only be provided by the class
  315. when it is being unpickled (removed from storage). Normal
  316. use is to call the class with no arguments and use the various
  317. addcounter functions (particularly, for end user's, the use of
  318. addcounterbybrowsing is the most common approach) You might
  319. want to provide the list directly if you want to hard-code the
  320. elements with which your query deals (and thereby avoid the
  321. overhead of unpickling the class).
  322. """
  323. self.volatilecounters = []
  324. BaseQuery.__init__(*(self,) + args, **namedargs)
  325. def addperfcounter(self, object, counter, machine=None):
  326. """
  327. A "Performance Counter" is a stable, known, common counter,
  328. such as Memory, or Processor. The use of addperfcounter by
  329. end-users is deprecated, since the use of
  330. addcounterbybrowsing is considerably more flexible and general.
  331. It is provided here to allow the easy development of scripts
  332. which need to access variables so common we know them by name
  333. (such as Memory|Available Bytes), and to provide symmetry with
  334. the add inst counter method.
  335. usage:
  336. query.addperfcounter('Memory', 'Available Bytes')
  337. It is just as easy to access addcounter directly, the following
  338. has an identicle effect.
  339. query.addcounter('Memory', 'Available Bytes')
  340. """
  341. BaseQuery.addcounter(self, object=object, counter=counter, machine=machine)
  342. def addinstcounter(
  343. self,
  344. object,
  345. counter,
  346. machine=None,
  347. objtype="Process",
  348. volatile=1,
  349. format=win32pdh.PDH_FMT_LONG,
  350. ):
  351. """
  352. The purpose of using an instcounter is to track particular
  353. instances of a counter object (e.g. a single processor, a single
  354. running copy of a process). For instance, to track all python.exe
  355. instances, you would need merely to ask:
  356. query.addinstcounter('python','Virtual Bytes')
  357. You can find the names of the objects and their available counters
  358. by doing an addcounterbybrowsing() call on a query object (or by
  359. looking in performance monitor's add dialog.)
  360. Beyond merely rearranging the call arguments to make more sense,
  361. if the volatile flag is true, the instcounters also recalculate
  362. the paths of the available instances on every call to open the
  363. query.
  364. """
  365. if volatile:
  366. self.volatilecounters.append((object, counter, machine, objtype, format))
  367. else:
  368. self.paths[len(self.paths) :] = self.getinstpaths(
  369. object, counter, machine, objtype, format
  370. )
  371. def getinstpaths(
  372. self,
  373. object,
  374. counter,
  375. machine=None,
  376. objtype="Process",
  377. format=win32pdh.PDH_FMT_LONG,
  378. ):
  379. """
  380. ### Not an end-user function
  381. Calculate the paths for an instance object. Should alter
  382. to allow processing for lists of object-counter pairs.
  383. """
  384. items, instances = win32pdh.EnumObjectItems(None, None, objtype, -1)
  385. # find out how many instances of this element we have...
  386. instances.sort()
  387. try:
  388. cur = instances.index(object)
  389. except ValueError:
  390. return [] # no instances of this object
  391. temp = [object]
  392. try:
  393. while instances[cur + 1] == object:
  394. temp.append(object)
  395. cur = cur + 1
  396. except IndexError: # if we went over the end
  397. pass
  398. paths = []
  399. for ind in range(len(temp)):
  400. # can this raise an error?
  401. paths.append(
  402. win32pdh.MakeCounterPath(
  403. (machine, "Process", object, None, ind, counter)
  404. )
  405. )
  406. return paths # should also return the number of elements for naming purposes
  407. def open(self, *args, **namedargs):
  408. """
  409. Explicitly open a query:
  410. When you are needing to make multiple calls to the same query,
  411. it is most efficient to open the query, run all of the calls,
  412. then close the query, instead of having the collectdata method
  413. automatically open and close the query each time it runs.
  414. There are currently no arguments to open.
  415. """
  416. # do all the normal opening stuff, self._base is now the query object
  417. BaseQuery.open(*(self,) + args, **namedargs)
  418. # should rewrite getinstpaths to take a single tuple
  419. paths = []
  420. for tup in self.volatilecounters:
  421. paths[len(paths) :] = self.getinstpaths(*tup)
  422. for path in paths:
  423. try:
  424. self.counters.append(win32pdh.AddCounter(self._base, path))
  425. self.curpaths.append(
  426. path
  427. ) # if we fail on the line above, this path won't be in the table or the counters
  428. except win32api.error:
  429. pass # again, what to do with a malformed path???
  430. def collectdatafor(self, totalperiod, period=1):
  431. """
  432. Non-threaded collection of performance data:
  433. This method allows you to specify the total period for which you would
  434. like to run the Query, and the time interval between individual
  435. runs. The collected data is stored in query.curresults at the
  436. _end_ of the run. The pathnames for the query are stored in
  437. query.curpaths.
  438. e.g.:
  439. query.collectdatafor(30,2)
  440. Will collect data for 30seconds at 2 second intervals
  441. """
  442. tempresults = []
  443. try:
  444. self.open()
  445. for ind in range(totalperiod / period):
  446. tempresults.append(self.collectdata())
  447. time.sleep(period)
  448. self.curresults = tempresults
  449. finally:
  450. self.close()
  451. def collectdatawhile(self, period=1):
  452. """
  453. Threaded collection of performance data:
  454. This method sets up a simple semaphor system for signalling
  455. when you would like to start and stop a threaded data collection
  456. method. The collection runs every period seconds until the
  457. semaphor attribute is set to a non-true value (which normally
  458. should be done by calling query.collectdatawhile_stop() .)
  459. e.g.:
  460. query.collectdatawhile(2)
  461. # starts the query running, returns control to the caller immediately
  462. # is collecting data every two seconds.
  463. # do whatever you want to do while the thread runs, then call:
  464. query.collectdatawhile_stop()
  465. # when you want to deal with the data. It is generally a good idea
  466. # to sleep for period seconds yourself, since the query will not copy
  467. # the required data until the next iteration:
  468. time.sleep(2)
  469. # now you can access the data from the attributes of the query
  470. query.curresults
  471. query.curpaths
  472. """
  473. self.collectdatawhile_active = 1
  474. _thread.start_new_thread(self.collectdatawhile_slave, (period,))
  475. def collectdatawhile_stop(self):
  476. """
  477. Signals the collectdatawhile slave thread to stop collecting data
  478. on the next logging iteration.
  479. """
  480. self.collectdatawhile_active = 0
  481. def collectdatawhile_slave(self, period):
  482. """
  483. ### Not a public function
  484. Does the threaded work of collecting the data and storing it
  485. in an attribute of the class.
  486. """
  487. tempresults = []
  488. try:
  489. self.open() # also sets active, so can't be changed.
  490. while self.collectdatawhile_active:
  491. tempresults.append(self.collectdata())
  492. time.sleep(period)
  493. self.curresults = tempresults
  494. finally:
  495. self.close()
  496. # pickle functions
  497. def __getinitargs__(self):
  498. return (self.paths,)
  499. def __getstate__(self):
  500. return self.volatilecounters
  501. def __setstate__(self, volatilecounters):
  502. self.volatilecounters = volatilecounters
  503. class QueryError:
  504. def __init__(self, query):
  505. self.query = query
  506. def __repr__(self):
  507. return "<Query Error in %s>" % repr(self.query)
  508. __str__ = __repr__