Comments on: JSON Performance in Python

By: Comparing Python JSON Implementations « Hustle Play

Comparing Python JSON Implementations « Hustle Play — Wed, 23 Dec 2009 12:19:03 +0000

[...] JSON Performance in Python [...]

By: Lloyd Hilaiel

Lloyd Hilaiel — Fri, 27 Nov 2009 19:43:29 +0000

@mikeal – throw py-yajl in this bucket as well if you like. @agentdero and I have recently collaborated for speed, and here’s what it currently looks like on my 64bit archlinux box (python 2.6.4) :

ParseTime: cjson 99.72539ms
ParseTime: yajl 141.28672ms
ParseTime: simplejson 223.61533ms
DumpTime: cjson 289.44809ms
DumpTime: yajl 154.87778ms
DumpTime: simplejson 192.81055ms

http://github.com/rtyler/py-yajl

cheers,
lloyd

By: John Millikin

John Millikin — Mon, 23 Nov 2009 16:49:45 +0000

I’m the author of jsonlib (which jsonlib2 forked off of)

First, please don’t use cjson for *anything*. It’s got multiple bugs and misfeatures, and is generally unsuited for anything except impressive benchmarks. It was easier to write my own library, from scratch, than try to fix cjson. To quote somebody, “If I want it done fast and wrong I’ll ask my cat”.

This also means you don’t have to support custom formatters in your libraries, because you don’t need to (and shouldn’t) support cjson.

Second, purely as a matter of curiosity, could you do an equivalent benchmark in Python 3? The version of jsonlib for Python 3 is even faster, because I got to drop some of the performance-degrading compatibility code.

By: Michal Chruszcz

Michal Chruszcz — Mon, 23 Nov 2009 09:07:19 +0000

This might be useful information:
>>> import json
>>> json.__version__
‘1.9′
while most important (influencing performance the most) changes in simplejson were added around 2.0 release.

The above chart might suggest that, in situation like yours, i.e. communication with database using JSON, when majority of operations is reading/decoding JSON, the best choice is cjson library. And this is true, with one caveat, though. The latest release of cjson – 1.0.5 – incorporates a bug in decoding escaped double quotes. Although it was never fixed by the maintainer of the library, there are some patches correcting this misbehaviour around, however I don’t know what’s their impact on performance.

By: mikeal

mikeal — Sat, 21 Nov 2009 18:46:02 +0000

I’m not a fan of monkey-patching the method names and it doesn’t solve the case where you want to use one module’s decoder and another modules encoder. Ideally, I would want to use cjson’s decoder and simplejson’s encoder so just having a module setting isn’t enough.

I’ve been using that try block since 2.6 was released but now that I’m using couchdb-pythonviews I’m trying to every ms of performance I can out of my json parser.

By: Bob Ippolito

Bob Ippolito — Sat, 21 Nov 2009 11:02:22 +0000

I think the no-monkeys-harmed answer to your question is:

try:
import simplejson as json
except ImportError:
import json

By: Leechael

Leechael — Sat, 21 Nov 2009 06:21:18 +0000

I’m have doubt why not you don’t use timeit module for benchmarking….

By: matt harrison

matt harrison — Sat, 21 Nov 2009 05:04:24 +0000

Maybe I’m not reading that last paragraph right, but couldn’t you either do some monkey patching to the module json and force its dump/load to actually use the fast code? Or create a json library that is just a wrapper that can do the try/except import dance for the fast apis and wrap those accordingly?

By: Python 的 JSON library 速度比較 at Gea-Suan Lin’s BLOG

Python 的 JSON library 速度比較 at Gea-Suan Lin’s BLOG — Sat, 21 Nov 2009 03:22:54 +0000

[...] 在「JSON Performance in Python」這篇看到比較圖，在 Python 2.6 上面測試 json (stdlib)、simplejson、jsonlib2、cjson 這四個不同的 JSON library 的速度。 [...]

By: Bob Ippolito

Bob Ippolito — Sat, 21 Nov 2009 00:35:37 +0000

The stdlib json is so far behind because they wouldn’t take the patch with all of the C speedups because it was too close to the release deadline. I also couldn’t get it in a point release since too much code changed and some behavior did too. There’s probably still something sitting in the tracker so hopefully it will make it in 2.7.