%M% %I% %E% The set of programs and documentation known as "lmbench" are distributed under the Free Software Foundation's General Public License with the following additional restrictions (which override any conflicting restrictions in the GPL): 1. You may not distribute results in any public forum, in any publication, or in any other way if you have modified the benchmarks. 2. You may not distribute the results for a fee of any kind. This includes web sites which generate revenue from advertising. If you have modifications or enhancements that you wish included in future versions, please mail those to me, Larry McVoy, at lm@bitmover.com. ========================================================================= Rationale for the publication restrictions: In summary: a) LMbench is designed to measure enough of an OS that if you do well in all catagories, you've covered latency and bandwidth in networking, disks, file systems, VM systems, and memory systems. b) Multiple times in the past people have wanted to report partial results. Without exception, they were doing so to show a skewed view of whatever it was they were measuring (for example, one OS fit small processes into segments and used the segment register to switch them, getting good results, but did not want to report large process context switches because those didn't look as good). c) We insist that if you formally report LMbench results, you have to report all of them and make the raw results file easily available. Reporting all of them means in that same publication, a pointer does not count. Formally, in this context, means in a paper, on a web site, etc., but does not mean the exchange of results between OS developers who are tuning a particular subsystem. We have a lot of history with benchmarking and feel strongly that there is little to be gained and a lot to be lost if we allowed the results to be published in isolation, without the complete story being told. There has been a lot of discussion about this, with people not liking this restriction, more or less on the freedom principle as far as I can tell. We're not swayed by that, our position is that we are doing the right thing for the OS community and will stick to our guns on this one. It would be a different matter if there were 3 other competing benchmarking systems out there that did what LMbench does and didn't have the same reporting rules. There aren't and as long as that is the case, I see no reason to change my mind and lots of reasons not to do so. I'm sorry if I'm a pain in the ass on this topic, but I'm doing the right thing for you and the sooner people realize that the sooner we can get on to real work. Operating system design is a largely an art of balancing tradeoffs. In many cases improving one part of the system has negative effects on other parts of the system. The art is choosing which parts to optimize and which to not optimize. Just like in computer architecture, you can optimize the common instructions (RISC) or the uncommon instructions (CISC), but in either case there is usually a cost to pay (in RISC uncommon instructions are more expensive than common instructions, and in CISC common instructions are more expensive than required). The art lies in knowing which operations are important and optmizing those while minimizing the impact on the rest of the system. Since lmbench gives a good overview of many important system features, users may see the performance of the system as a whole, and can see where tradeoffs may have been made. This is the driving force behind the publication restriction: any idiot can optimize certain subsystems while completely destroying overall system performance. If said idiot publishes *only* the numbers relating to the optimized subsystem, then the costs of the optimization are hidden and readers will mistakenly believe that the optimization is a good idea. By including the publication restriction readers would be able to detect that the optimization improved the subsystem performance while damaging the rest of the system performance and would be able to make an informed decision as to the merits of the optimization. Note that these restrictions only apply to *publications*. We intend and encourage lmbench's use during design, development, and tweaking of systems and applications. If you are tuning the linux or BSD TCP stack, then by all means, use the networking benchmarks to evaluate the performance effects of various modifications; Swap results with other developers; use the networking numbers in isolation. The restrictions only kick in when you go to *publish* the results. If you sped up the TCP stack by a factor of 2 and want to publish a paper with the various tweaks or algorithms used to accomplish this goal, then you can publish the networking numbers to show the improvement. However, the paper *must* also include the rest of the standard lmbench numbers to show how your tweaks may (or may not) have impacted the rest of the system. The full set of numbers may be included in an appendix, but they *must* be included in the paper. This helps protect the community from adopting flawed technologies based on incomplete data. It also helps protect the community from misleading marketing which tries to sell systems based on partial (skewed) lmbench performance results. We have seen many cases in the past where partial or misleading benchmark results have caused great harm to the community, and we want to ensure that our benchmark is not used to perpetrate further harm and support false or misleading claims.