tools/trace-malloc/README@97036ab72558 (annotated)
tools/trace-malloc/README
Tue, 06 Jan 2015 21:39:09 +0100
- author
- Michael Schloh von Bennewitz <michael@schloh.com>
- date
- Tue, 06 Jan 2015 21:39:09 +0100
- branch
- TOR_BUG_9701
- changeset 8
- 97036ab72558
- permissions
- -rw-r--r--
Conditionally force memory storage according to privacy.thirdparty.isolate;
This solves Tor bug #9701, complying with disk avoidance documented in
https://www.torproject.org/projects/torbrowser/design/#disk-avoidance.
| michael@0 | 1 | Trace Malloc Tools |
| michael@0 | 2 | Chris Waterson <waterson@netscape.com> |
| michael@0 | 3 | November 27, 2000 |
| michael@0 | 4 | |
| michael@0 | 5 | This is a short primer on how to use the `trace malloc' tools |
| michael@0 | 6 | contained in this directory. |
| michael@0 | 7 | |
| michael@0 | 8 | |
| michael@0 | 9 | WHAT IS TRACE MALLOC? |
| michael@0 | 10 | ===================== |
| michael@0 | 11 | |
| michael@0 | 12 | Trace malloc is an optional facility that is built in to XPCOM. It |
| michael@0 | 13 | uses `weak linking' to intercept all calls to malloc(), calloc(), |
| michael@0 | 14 | realloc() and free(). It does two things: |
| michael@0 | 15 | |
| michael@0 | 16 | 1. Writes information about allocations to a filehandle that you |
| michael@0 | 17 | specify. As each call to malloc(), et. al. is made, a record is logged |
| michael@0 | 18 | to the filehandle. |
| michael@0 | 19 | |
| michael@0 | 20 | 2. Maintains a table of all `live objects' -- that is, objects that |
| michael@0 | 21 | have been allocated by malloc(), calloc() or realloc(), but have not |
| michael@0 | 22 | yet been free()'d. The contents of this table can be called by making |
| michael@0 | 23 | a `secret' call to JavaScript. |
| michael@0 | 24 | |
| michael@0 | 25 | |
| michael@0 | 26 | MAKING A TRACE MALLOC BUILD |
| michael@0 | 27 | =========================== |
| michael@0 | 28 | |
| michael@0 | 29 | As of this writing, trace malloc only works on Linux, but work is |
| michael@0 | 30 | underway to port it to Windows. |
| michael@0 | 31 | |
| michael@0 | 32 | On Linux, start with a clean tree, and configure your build with the |
| michael@0 | 33 | following flags: |
| michael@0 | 34 | |
| michael@0 | 35 | --enable-trace-malloc |
| michael@0 | 36 | --enable-cpp-rtti |
| michael@0 | 37 | |
| michael@0 | 38 | Be sure that `--enable-boehm' is *not* set. I don't think that the |
| michael@0 | 39 | values for `--enable-debug' and `--enable-optimize' matter, but I've |
| michael@0 | 40 | typically had debug on and optimize off. |
| michael@0 | 41 | |
| michael@0 | 42 | |
| michael@0 | 43 | COLLECTING LIVE OBJECT DATA |
| michael@0 | 44 | =========================== |
| michael@0 | 45 | |
| michael@0 | 46 | To collect `live object' data from `mozilla' using a build that has |
| michael@0 | 47 | trace malloc enabled, |
| michael@0 | 48 | |
| michael@0 | 49 | 1. Run `mozilla' as follows: |
| michael@0 | 50 | |
| michael@0 | 51 | % mozilla --trace-malloc /dev/null |
| michael@0 | 52 | |
| michael@0 | 53 | 2. Do whatever operations in mozilla you'd like to test. |
| michael@0 | 54 | |
| michael@0 | 55 | 3. Open the `live-bloat.html' file contained in this directory. |
| michael@0 | 56 | |
| michael@0 | 57 | 4. Press the button that says `Dump to /tmp/dump.log' |
| michael@0 | 58 | |
| michael@0 | 59 | An enormous file (typically 300MB) called `dump.log' will be dropped |
| michael@0 | 60 | in your `/tmp' directory. |
| michael@0 | 61 | |
| michael@0 | 62 | To collect live object data from `gtkEmbed' using a build that has |
| michael@0 | 63 | trace malloc enabled: |
| michael@0 | 64 | |
| michael@0 | 65 | 1. Run `gtkEmbed' as follows: |
| michael@0 | 66 | |
| michael@0 | 67 | % gtkEmbed --trace-malloc /dev/null |
| michael@0 | 68 | |
| michael@0 | 69 | 2. Do whatever operations in gtkEmbed that you'd like to test. |
| michael@0 | 70 | |
| michael@0 | 71 | 3. Press the `Dump Memory' button at the bottom of gtkEmbed. |
| michael@0 | 72 | |
| michael@0 | 73 | The enormous file will be dropped in the current directory, and is |
| michael@0 | 74 | called `allocations.log'. |
| michael@0 | 75 | |
| michael@0 | 76 | |
| michael@0 | 77 | About Live Object Logs |
| michael@0 | 78 | ---------------------- |
| michael@0 | 79 | |
| michael@0 | 80 | A typical entry from the `live object' dump file will look like: |
| michael@0 | 81 | |
| michael@0 | 82 | Address Type Size |
| michael@0 | 83 | | | | |
| michael@0 | 84 | v v v |
| michael@0 | 85 | 0x40008080 <nsFooBar> 16 |
| michael@0 | 86 | 0x00000001 <- Fields |
| michael@0 | 87 | 0x40008084 |
| michael@0 | 88 | 0x80004001 |
| michael@0 | 89 | 0x00000036 |
| michael@0 | 90 | __builtin_new[./libxpcom.so +0x10E9DC] <- Stack at allocation time |
| michael@0 | 91 | nsFooBar::CreateFooBar(nsFooBar **)[./libfoobar.so +0x408C] |
| michael@0 | 92 | main+C7E5AFB5[(null) +0xC7E5AFB5] |
| michael@0 | 93 | |
| michael@0 | 94 | One will be printed for each object that was allocated. |
| michael@0 | 95 | |
| michael@0 | 96 | |
| michael@0 | 97 | TOOLS TO PARSE LIVE OBJECT LOGS |
| michael@0 | 98 | =============================== |
| michael@0 | 99 | |
| michael@0 | 100 | This directory is meant to house the tools that you can use to parse |
| michael@0 | 101 | live-object logs. |
| michael@0 | 102 | |
| michael@0 | 103 | Object Histograms - histogram.pl |
| michael@0 | 104 | -------------------------------- |
| michael@0 | 105 | |
| michael@0 | 106 | This program parses a `live object' dump and produces a histogram of |
| michael@0 | 107 | the objects, sorted from objects that take the most memory to objects |
| michael@0 | 108 | that take the least. The output of this program is rather spartan: on |
| michael@0 | 109 | each line, it prints the object type, the number of objects of that |
| michael@0 | 110 | type, and the total number of bytes that the objects consume. |
| michael@0 | 111 | |
| michael@0 | 112 | There are a two simple programs to `pretty print' the output from |
| michael@0 | 113 | histogram.pl: |
| michael@0 | 114 | |
| michael@0 | 115 | 1. histogram-pretty.sh takes a single histogram and produces a table |
| michael@0 | 116 | of objects. |
| michael@0 | 117 | |
| michael@0 | 118 | Type Count Bytes %Total |
| michael@0 | 119 | TOTAL 67348 4458127 100.00 |
| michael@0 | 120 | nsImageGTK 76 679092 15.23 |
| michael@0 | 121 | void* 8956 563572 12.64 |
| michael@0 | 122 | ... |
| michael@0 | 123 | PRLock 732 61488 1.38 |
| michael@0 | 124 | OTHER 24419 940235 21.09 |
| michael@0 | 125 | |
| michael@0 | 126 | 2. histogram-diff.sh takes two histograms and computes the difference |
| michael@0 | 127 | between them. |
| michael@0 | 128 | |
| michael@0 | 129 | ---- Base ---- ---- Incr ---- ----- Difference ---- |
| michael@0 | 130 | Type Count Bytes Count Bytes Count Bytes %Total |
| michael@0 | 131 | TOTAL 40241 1940945 73545 5315142 33304 3374197 100.00 |
| michael@0 | 132 | nsImageGTK 16 106824 151 832816 135 725992 21.52 |
| michael@0 | 133 | PresShell 16 51088 198 340706 182 289618 8.58 |
| michael@0 | 134 | ... |
| michael@0 | 135 | OTHER 27334 1147033 38623 1493385 11289 346352 10.26 |
| michael@0 | 136 | |
| michael@0 | 137 | Both of these scripts accept `-c' parameter that specifies how many |
| michael@0 | 138 | rows you'd like to see (by default, twenty). Any rows past the first |
| michael@0 | 139 | `n' rows are lumped into a single `OTHER' row. This allows you to keep |
| michael@0 | 140 | your reports short n' sweet. |
| michael@0 | 141 | |
| michael@0 | 142 | Stack-based Type Inference - types.dat |
| michael@0 | 143 | -------------------------------------- |
| michael@0 | 144 | |
| michael@0 | 145 | Trace malloc uses `speculative RTTI' to determine the types of objects |
| michael@0 | 146 | as it dumps them. Unfortunately, RTTI can only deduce the type name |
| michael@0 | 147 | for C++ objects with a virtual destructor. |
| michael@0 | 148 | |
| michael@0 | 149 | This leaves: |
| michael@0 | 150 | |
| michael@0 | 151 | . C++ object without a virtual destructor |
| michael@0 | 152 | . array allocated C++ objects, and |
| michael@0 | 153 | . objects allocated with the C runtime function (malloc |
| michael@0 | 154 | and friends) |
| michael@0 | 155 | |
| michael@0 | 156 | out in the cold. Trace malloc reports objects allocated this was as |
| michael@0 | 157 | having type `void*'. |
| michael@0 | 158 | |
| michael@0 | 159 | The good news is that you can almost always determine the object's |
| michael@0 | 160 | type by looking at the stack trace that's taken at the time the object |
| michael@0 | 161 | is allocated. |
| michael@0 | 162 | |
| michael@0 | 163 | The file `types.dat' consists of rules to classify objects based on |
| michael@0 | 164 | stack trace. |
| michael@0 | 165 | |
| michael@0 | 166 | |
| michael@0 | 167 | Uncategorized Objects - uncategorized.pl |
| michael@0 | 168 | ---------------------------------------- |
| michael@0 | 169 | |
| michael@0 | 170 | Categorizing objects in `types.dat' is sweaty work, and the |
| michael@0 | 171 | `uncategorized.pl' script is a tool that makes it a bit |
| michael@0 | 172 | easier. Specifically, it reads a `live object' dump file and sorts the |
| michael@0 | 173 | stack traces. Stack traces that account for the most uncategorized |
| michael@0 | 174 | objects are placed first. |
| michael@0 | 175 | |
| michael@0 | 176 | Using this tool, you can add the `most effective' rules to |
| michael@0 | 177 | `types.dat': rules that account for most of the uncategorized data. |
