Tuesday, June 29, 2010

WebYaST: Switch from XML into JSON (day one)

This week I'll continue on project that started on recent WebYaST Workshop:WebYaST_Performance

Result from previous tests was that bundled reXML parser is slowest from all ;-)
Now I'd like to continue with comparison XML and JSON performance. For this purpose I created webclient json branch in our git repository.


To install profiling extension you'll need to add repository http://download.opensuse.org/repositories/devel:/languages:/ruby:/extensions/openSUSE_11.2
and install package rubygem-ruby-prof. Also to see results from profiling you'll need kcachegrind package.

I started some tests with Firefox firebug extension (5 measurements from each and then calculate average). First one without any modification, then with profiling enabled.

xml (rexml), no benchmarkingxml (rexml), ruby-prof enabled
5.536.3
6.535.71
6.155.79
6.555.78
6.155.99
6.1825.914







On this picture you can see (selected line is xml parsing process) with luck we can speed up 1/5 times (about 1 sec in my case). There's almost no difference between creating json and xml output: Calling http://localhost:4984/groups.xml takes average time 655ms, http://localhost:4984/groups.json takes average time 621.4ms.
In rest-service there are some problems - json output is not valid, jreidinger is working on it.
I'll continue tomorrow.

5 comments:

Garrett said...

Hi!

Are you guys using a JSON library of some sort, or Rails-provided toJSON?

(You shouldn't hand-write JSON, as it's too easy to produce broken JSON, and browsers — especially some — will stop working if a comma is missing or if things are not quoted properly.)

But overall, awesome news! (:

Pavol Rusnak said...

In OBS we use rubygem-libxml-ruby not reXML. Also XML has an advantage that you can validate the messages. If this is not needed then JSON is probably the better way.

Sandy said...

When working on yastroid, we found that not every webyast API supports JSON. Some only supported XML.

JSON is way easier to consume, so I am happy at anything that might lead to more consistent JSON support in webyast! :-)

Michal Žugec said...

Garrett: we're using "Rails-provided toJSON" with some hacks for Arrays and Hashes

Pavol: yes, I tried libxml and nokogiri with some performance speed up. But complete switch into JSON can improve it more

Sandy: wow, I didn't know about yastroid until now - it seems cool!
If we decide to switch into JSON we'll retest all plugins but if you know about any particular one, write bugreport please.

Josef Reidinger said...

Garrett - problem is quite hidden as we use json from rails, but not so much good as we parse xml and then serve json, which I plan to change and now almost all plugins use new behavior for serialization of our model.

Sandy - We don't know about it until miso try it. If you found it during work on yastroid you should report it to bugzilla or at least mail to our mailing list yast-devel@opensuse.org.