MirrorBand

From CSLabsWiki
Jump to: navigation, search

The MirrorBand project focuses on collecting and then analyzing data from Mirror. This currently spans two main inputs, logs created by dstat and the Nginx access logs.

MirrorBand

MirrorBand is a series of various programs which generate the data for most of the charts and figures on the main statistics page. The main programs are written in Golang and can be found at lannonbr/MirrorBandwidthStats. The data is collected on Mirror, sent to Norm, and analyzed and saved in various formats to a SQLite database.

Every hour, the data is collected using dstat and then sent over to Norm. Norm a few minutes later then parses it and inserts the new data and modifies relevant rows in the tables. Following, a subprogram MBExporter sends a series of queries off to the database and generates JSON files which are sent back to Mirror.

Finally, when a user loads the statistics webpage, it will use the JSON and render out the graphs and figures using SVG and C3.js.

DistroUsage

This portion focuses on examining the Bandwidth across all repos hosted on Mirror. A file is generated daily at 12AM for the statistics about the previous day.

On the stats page, there should be a link near the bottom consisting of both the latest file as well as an archive of all previous files.