Amazingly huge performance regression

steviant's Avatar

steviant

12 Jun, 2015 12:30 AM

Hi, I keep the last version of Tobias' "Aurorafox" builds of Firefox for 10.5 that supports plugins around, just in case I need to use flash or java on a known safe website.

Just for a laugh I decided to compare the results of the V8 (version 7) benchmark between Aurorafox 17.0.2 and TenFourFox 31.something, and found an astonishing difference in performance:

Aurorafox: 1058
TenFourFox: 246

Just to be clear, Aurorafox benchmarks more than four times faster than TenFourFox. I was expecting a small but appreciable increase in performance, rather than a regression to less than 25% of the performance from two years ago..

Needless to say, I've reversed my preference between these browsers and will keep browsing like it's 2013 with Aurorafox unless I see some major improvements in performance, but it seems unlikely that you're going to find a way to increase the speed of TenFourFox by four times, in fact it's hard to believe that performance could regress so far.

  1. Support Staff 1 Posted by Chris (chtrusch... on 12 Jun, 2015 04:39 AM

    Chris (chtrusch)'s Avatar

    Hello steviant, I also see this difference on V8 between TFF 17 and 31. Now try this one: https://www.webkit.org/perf/sunspider/sunspider.html :-) And then do a few real world tests like loading the Facebook timeline or Ebay search results or the Amazon start page, and decide for yourself whether 17 is really 4 times faster than 31.

  2. 2 Posted by steviant on 12 Jun, 2015 04:59 AM

    steviant's Avatar

    Chris, I realise that benchmarks are not the be-all and end-all of performance testing, and that TenFourFox is not really running at a quarter of the speed of Aurorafox most of the time, but the benchmark is highlighting some pretty significant performance regressions; showing that in some tests (richards for example) TenFourFox is more than one thousand percent slower, surely this can't have zero effect on overall performance

    Neither using a benchmark that doesn't test that particular code-path, nor testing responsiveness side-by-side does anything to mitigate the fact that there is a significant regression in performance doing whatever it is that v8 does to test the browser. It may not make much difference in the real world, but I would have thought that such a remarkable regression would warrant some attention and not a hand-waving dismissal.

  3. 3 Posted by steviant on 12 Jun, 2015 05:28 AM

    steviant's Avatar

    For what it's worth Sunspider agrees that there has been a huge performance regression, though not quite to the same extent.

    Aurorafox completes Sunspider 1.0.2 in 1797.0ms, while TenFourFox takes 3245.3ms, so Aurorafox is a mere 1.8x faster than TenFourFox according to Sunspider. /s

  4. Support Staff 4 Posted by Chris (chtrusch... on 12 Jun, 2015 05:30 AM

    Chris (chtrusch)'s Avatar

    Yes, TFF 17 was faster on some JS operations than 24/29/31, and (being a fan of these benchmarks myself) I can feel your pain. The JS engine has changed dramatically between 17 and 31 (several times), and my understanding is that it hasn't even been fully implemented for 31 because it is harder and harder to keep up with Firefox, which is not optimized for PPC anymore.

    But all this, amazingly, doesn't translate into real world surfing experience most of the time. On TFF 17, Facebook is dog slow, while on 31, most pages I visit regularly are snappy. [The only time in several months that I wished for faster JS was when I did my taxes (which can be done online in Germany), and TFF had to generate two 1024 bit keys for authentication. Which took 20 minutes and would probably have been faster on TFF 17. Only that older browsers aren't allowed to log-in to the taxes website anymore for security reasons…]

    BTW, on Sunspider, TFF 17 is now "twice" as fast as 31, not 4 times. So which one is right? It all depends on what you measure.

  5. Support Staff 5 Posted by Cameron Kaiser on 12 Jun, 2015 02:21 PM

    Cameron Kaiser's Avatar

    steviant, this isn't news. When Mozilla ended support for the old JaegerMonkey JIT in version 22 (AuroraFox is frozen around 20), we had no choice but to stop using it as well. IonMonkey, which is implemented as IonPower in the upcoming TenFourFox 38, is somewhat faster, but it's optimized for longer running code and is much more tuned to Mozilla's primary Tier-1 platforms of x86 and ARM. Other than marginal improvements, the numbers will improve by about double on the tests you're using, but probably not much more than that.

    If AuroraFox is meeting your needs, use it, and good luck.

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Already uploaded files

  • regression.png 141 KB

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac