About
Mmstats is a way to expose and read diagnostic values and metrics for
applications.
Think of mmstats as /proc for your application and the readers as procps
utilities.
This project is a Python implementation, but compatible implementations can be
made in any language (see Goals).
Discuss at https://groups.google.com/group/python-introspection
Goals
- Separate publishing/writing from consuming/reading tools
- Platform/language independent (a Java writer can be read by a Python tool)
- Predictable performance impact for writers via:
- No locks (1 writer per thread)
- No syscalls (after instantiation)
- All in userspace
- Reading has no impact on writers
- Optional persistent (writer can sync anytime)
- 1-way (Publish/consume only; mmstats are not management extensions)
Usage
Requirements
CPython 2.6 or 2.7 (Windows is untested)
PyPy (only tested in 1.7, should be faster in 1.8)
Using
- easy_install mmstats or pip install mmstats or if you’ve downloaded
the source: python setup.py install
- Then in your Python project create a sublcass of mmstats.MmStats like
import mmstats
class WebStats(mmstats.MmStats):
status2xx = mmstats.CounterField(label='status.2XX')
status3xx = mmstats.CounterField(label='status.3XX')
status4xx = mmstats.CounterField(label='status.4XX')
status5xx = mmstats.CounterField(label='status.5XX')
last_hit = mmstats.DoubleField(label='timers.last_hit')
- Instantiate it once per process: (instances are automatically thread local)
webstats = WebStats(label_prefix='web.stats.')
- Record some data:
if response.status_code == 200:
webstats.status2xx.inc()
webstats.last_hit = time.time()
- Run slurpstats to read it
- Run mmash to create a web interface for stats
- Run pollstats -p web.stats.status 2XX,3XX,4XX,5XX /tmp/mmstats-* for a
vmstat/dstat like view.
- Did a process die unexpectedly and leave around a stale mmstat file?
cleanstats /path/to/mmstat/files will check to see which files are stale
and remove them.
History
0.8.0
- MmStats.remove now removes all files for a PID if you use the TID in the
filenames (thanks @bitprophet!)
- mmash now prepends MMSTATS_PATH to ?glob=... parameters so users
can’t traverse the filesystem.
- Javascript fixes for mmash‘s graphs (thanks @spiccinini!)
- Handle broken pipes in slurpstats
0.7.2 “Mr. Clean” released 2012-12-12
- cleanstats now cleans mmstats files for alive PIDs that are owned by other
users.
- Minor cleanups to tests
0.7.1 “Mash Tun” released 2012-11-19
- cleanstats now defaults to DEFAULT_GLOB if no files are passed on the
command line
- CounterField.inc(n=...) has been deprecated in favor of
CounterField.incr(amount=...)
- mmash Improvements
- Added nonzero-avg and nonzero-min aggregators to mmash to filter 0s
out of aggregated metrics.
- Added a glob query parameter to /stats/<stats> to filter which
mmstats files are included in the stats
- Backward incompatible change: switched /stats/ to return a JSON Object
instead of an array. The array is now the value of the stats key.
- Minor documentation and code cleanups
0.7.0 “Local Artisanal Stats” released 2012-10-02
- Per-thread model instances are created automatically - no need to manually
create one per thread
- Backward incompatible change to naming templates; they now use str.format
style substitutions:
- New: {CMD} - Current process name
- %PID% -> {PID}
- %TID% -> {TID}
0.6.2 “Graphtastic” released 2012-03-23
- Added live graphing of numeric metrics thanks to @haard’s work at PyCon
- Documentation improvements
0.6.1 “MANIFEST.out” released 2012-03-08
0.6.0 “PyCon 2012” released 2012-03-08
- [API CHANGE] - MovingAverageField’s kwarg changed from window_size => size
- Refactored __init__.py into fields, models, and default (and imported public
bits into __init__)
- Added TimerField (MovingAverageField + context manager)
- Added docs (don’t get too excited, just a start)