HOME
Item Database
Chronicles
DEV BLOGS
EVE Gate
FORUMS

Mass testing results

Article Discussion View source History

Contents

Below you can find the "official" results of past mass-tests. For general feedback you can use the Discussion page, but please use the forums for discussing single mass-tests.

Mass-tests in 2012

Thursday, March 29th

Goals:

  • Test client performance
  • Test fixes for client crashes
  • Test fix for client clock drift

Results:

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~180 18 30.4 6.1
  • Very low player participation (possible causes: different time, on Duality, late announcement)
  • Good client performance and low amount of crashes - probably related to the low player participation
  • Client clock drift only partially fixed (the computer clock is still drifting), no problems found

Thursday, March 8th

Goals:

  • Testing bracket and overview performance improvements
  • Investigating client crashes
  • Testing new EVE Launcher
  • Data gathering for various bugs

Results:

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~440 14.5 19.3 6.0
  • Good server performance and quite good client performance
  • Some problems with the EVE Launcher
  • Clients are still crashing
  • Data gathered for the "client clock drift", which was experienced by some players

Thursday, March 1st

Goals:

  • Testing bracket performance improvements
  • Investigating client crashes
  • Testing new EVE Launcher
  • Data gathering for various bugs

Results:

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~590 10.3 20.1 4.6
  • Good server performance (for this amount of players - we are back on the old server hardware)
  • OK-ish client performance (still worse than server, but much better than 2 months ago)
  • Many client crashes
  • Many bugs
  • Launcher worked for most (but not all) players, Mac broken

Tuesday, February 14th

Goals:

  • Testing overview performance improvements

Results:

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~320 19.4 27.7 5.8
  • UI performance improvements are good
  • Low server performance (caused by hardware changes)

Tuesday, February 7th

Goals:

  • Testing overview performance improvements
  • Testing bracket performance improvements

Results:

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~330 12.1 20.5 4.6
  • Quite good client performance
  • Several UI bugs
  • Low server performance (caused by hardware changes)

Thursday, January 26th

Goals:

  • Investigating client crashes
  • Testing server code improvements

Results:

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~280 12.4 19.4 5.4
  • Not as many client crashes (but also less players) - no clear cause found

Thursday, January 12th

Goals:

  • Testing general performance and stability of Crucible 1.2
  • Testing the fleet watchlist

Results:

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~420 12.4 19.4 5.4
  • No problems with the fleet watchlist
  • High amount of client crashes
  • High memory usage of the client?

Mass-tests in 2011

Thursday, December 22nd

Goals:

  • Special test for Time Dilation

Results:

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~250 x x x
  • No issues found

Thursday, December 15th

Goals:

  • Time Dilation

Results:

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~340 11.5 19.9 4.7
  • More bugs found for Time Dilation (when jumping between differently dilated systems)

Thursday, December 8th

Goals:

  • Time Dilation

Results:

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~620 5.9 13.9 4.5
  • Bugs identified for Time Dilation
  • High player count
  • Docking and undocking is causing a lot of lag
  • Low client performance
  • Increased rate of client crashes

Friday, December 2nd

Goals:

  • Time Dilation
  • Overview window optimizations
  • Remapping of a system to another server node

Results:

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~220 21.0 34.6 6.9
  • No problems with overview changes, but too low player participation for usable performance metrics
  • Bugs identified for TIme Dilation
  • No problems with node remap

Thursday, November 17th

Goals:

  • Performance of the Crucible client and server
  • Remapping of a system to another server node
  • Stacking of items

Results:

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~600 5.7 11.9 4.1
  • Low client performance
  • No problems with stacking of items
  • Another slow remap (logs gathered and cause found)

Thursday, November 10th

Goals:

  • Time Dilation
  • Remapping of a system to another server node

Results:

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~530 6.3 15.6 4.8
  • no problems with Time Dilation
  • Remapping of a system worked but a bit slow
  • Client performance quite low (many players, Time Dilation was allowed to sped up the game)

Thursday, October 20th

Goals:

  • Time Dilation
  • Remapping of a system to another server node
  • Stacking of items

Results:

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~420 8.9 16.9 5.8
  • Time Dilation worked successfully (the balancing needs a bit fine-tuning)
  • Remapping of a system worked, but minor issues to be investigated

Graph of the CPU usage of the server node and the Time Dilation factor during this mass test: Masstest 20 Oct 2011 FD-MLJ TiDi.png

Saturday, October 1st

Goals:

  • Time Dilation
  • Several other new features
  • DB fail-over

Results:

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~300 18.6 29.3 6.0

Please note: The FPS values from this mass-test are not comparable with other mass-tests as there was no fight at a gate.

  • Time dilation worked!
  • Massive amount of bugs (one or two were caused by time dilation)
  • Several node crashes (already fixed internally)
  • DB fail-over produced good results
  • Server had to be restarted after the test (node deaths + problems after fail-over)

Thursday, September 29th

Goals:

  • Checking for Mac client crashes

Results:

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~350 10.0 17.6 5.3
  • Gathered logs and dumps of several Mac client crashes - unrelated to CarbonIO
  • EveVoice broken

Thursday, September 22nd

Goals:

  • Testing CarbonIO on the client
  • Checking for Mac client crashes

Results:

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~250 11.0 19.4 5.8
  • Mac clients are still crashing very frequently - perhaps a bit less than in the tests in August
  • Low participation (causes: mass-test announced too late, no mail sent to mailing list, focused on Macs)

Thursday, September 8th

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~350 8.9 16.1 4.5
  • One reason for crashing Mac clients was already identified before the mass-test
    • Loading a high amount of (agent) character portraits at the same time causes crashes of the Mac client in combination of CarbonIO enabled in the client
  • During the mass-test: Reduced but still high rate of Mac client crashes (by using a workaround to avoid the identified crash)
  • Telemetry data of mass-undock lag gathered, but without finding unexpected problems
    • There were complaints about the bad user experience of only showing a black screen while waiting to undock

Thursday, August 25th

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~450 8.7 16.2 4.0
  • Still a very high amount of Mac client crashes
    • In-house tests after the mass-test showed, that at least a part of the Mac crashes were probably caused by CarbonIO on the client. As a result of this CarbonIO on the client was disabled again for the deployment of the Incarna 1.1 release.
  • A low client performance was recorded and reported by players
  • The overview showed much less problems than in the last two mass-tests
  • No problems with the server performance (except during the mass-undock and when killing hundreds of ships with a smartbomb)

Thursday, August 18th

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~460 8.7 18.8 4.5
  • Most bugs from the mass-test on the 9th are gone
    • Still some problems caused by the new undock process, for example after being podded
  • Massive performance issues with the overview
  • High amount of Mac client crashes reported
  • The mass-undock was very heavy on the server

Tuesday, August 9th

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~460 10.8 21.3 4.2
  • Massive bugs, mostly related to the new undock process and the overview
    • The overview was nearly unusable due to many duplicate entries
    • Many bugs with offline modules, caused by the new undock process
  • High amount of Mac client crashes, but low amount of Windows client crashes
  • One server node crashed due to a known problem on test-servers, caused the markets to be offline for some time

Thursday, June 16th

Goals:

  • In-space parts of the Incursion client and server (including Dogma refactoring and the new turret graphics)

Results:

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~450 9.1 17.1 4.6
  • Good server performance
  • No major problems
  • Low client performance (low FPS, high memory usage
  • Long black screen after undocking

Tuesday, June 7th

Goals:

  • Testing the Dogma refactoring (the system, which handles for example all module attributes)
  • Testing the new turret graphics

Results:

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~430 13.8 34.6 5.2
  • Fast test without major problems

Tuesday, May 24th

Goals:

  • CarbonUI
  • CarbonIO
  • gathering Telemetry data

Results:

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~380 13.3 27.3 5.3
  • Smooth test with no major problems
  • Minor bugs with CarbonUI
  • No problems with CarbonIO
  • Good data from Telemetry

Thursday, May 12th

Goals:

  • gathering performance data with Telemetry

Results:

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~360 8.0 12.3 5.7
  • Very good data from Telemetry
  • One node crash

Thursday, April 28th

Goals:

  • CarbonUI
  • CarbonIO
  • Remapping of systems to a different server node

Results:

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~390 12.5 24.9 5.1
  • Many bugs with CarbonUI, including many client crashes
  • Several node crashes, caused by CarbonIO
  • Remapping the system failed - the system did not load correctly on the new node

Thursday, April 14th

Goals:

  • Remapping of systems to a different server node
  • Improvements to weapon grouping
  • Scanning improvements

Results:

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~260 15.6 30.0 5.8
  • Good and productive test, but with only 260 participants
  • Remap test failed
  • No problems with improvements from Team Gridlock

Thursday, March 31st

Goals:

  • Tests for low sec starbase improvements
  • Testing other performance improvements

Results:

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~390 11.3 25.2 5.7
  • No major problems, good server perfomance
  • Low client performance during POS bash
  • One node crash
  • Minor interruptions from protests against anomaly changes

Thursday, March 17th

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~130 17.3 33.0 5.7
  • Again very low player count (similar reasons as on the 10th)
  • First mass test on the new test-server Duality (this also reduced the player count)
  • Low sec POS bash performed better than the one on the 3rd Feb. (but difficult to compare due to low player count and different server)

Thursday, March 10th

Number of participants Average FPS, zoomed in Average FPS, zoomed out Average Player rating
~140 14.5 29.8 5.7
  • Very low player count, due to:
    • Huge patches
    • Server available too late
    • Many bugs
  • Remapping had to be canceled (because of a bug, which was found shortly before the test)
  • no server lag (not enough players)

Mass-tests in 2010

Saturday, February 20th 2010

Number of participants Average FPS Average Player rating
~400 21.8 5.2
  • During the course of this test; the following issues were reported, which we will be following up on:
    • Players report very-low FPS in some cases, even on high-end machines
    • Fleet-swapping may add load to nodes
    • Fleets, or being in one, seems to cause the most noticeable lag/FPS hit for players
  • More details:
    • We were able to get around 400 players for this test. The test involved two fleets, ‘campers’ and ‘attackers’, one jumping into the other. We were able to use FleetFinder ads to bring players into fleets much more easily compared to previous test events. We were able to test jumping between 2 dedicated nodes as well as from a dedicated node to a shared node. Almost no lag was reported when jumping a 200-man fleet into another of roughly equal size; however, when jumping 400 players at once and if combat is happening on the destination node, there came reports of “no-load” issues. Combat occurring caused the most noticeable difference, by far. In addition to this, we also ran a brief test of fleet-swapping; meaning having an entire fleet drop and join a new fleet en-masse. This last test showed that there is a little extra load when large numbers of people join a new fleet, we will be following up on this internally.

Older reports

Read this old forum thread

Retrieved from "http://wiki.eveonline.com/en/wiki/Mass_testing_results"