Allegro 5 regression tests

This script automatically runs once a day. It updates the source code, builds it, then runs all the tests.

Linux

For testing the Linux binaries I'm compiling to native 64-bit code then run it using the xvfb X11-server with Mesa software OpenGL. This means some tests are not very representative - but it should give a good indication.

Windows

For testing the Windows binaries I'm cross-compiling to 32-bit code using the MingW compiler, then run Wine inside the xvfb X11 server (with Mesa software OpenGL). Again this means the tests are not very representative - but we still know that Wine's DirectX implementation running on Mesa's OpenGL implementation draws Allegro graphics correctly.

OSX

For testing the OSX version it is compiled and run inside an OSX 10.7 VirtualBox instance. This means we either test the VirtualBox or again the Mesa software renderer. Still, a successful test here means that the OSX version displays correctly under these conditions.

Other

Ideally we would have more test machines of course, maybe somehow run iOS and Android tests and then for different GPUs like one for NVidia, AMD and Intel each...

All days where the exact same tests failed/succeeded are omitted below to not make the list too long while still keeping historic tests.

datetestspassedfailedreportcomments
2017-08-17 OSX 981 978 3 report raw data
                                                                                                   
 
2017-08-17 Windows 981 969 12 report raw data
                                                                                                  
  
2017-08-17 Linux 981 981 0 report raw data
                                                                                                    
2017-06-22 OSX 981 911 70 report raw data
                                                                                            
        
2017-06-20 OSX 981 978 3 report raw data
                                                                                                   
 
2017-06-04 OSX 981 911 70 report raw data
                                                                                            
        
2017-06-01 OSX 981 978 3 report raw data
                                                                                                   
 
2017-05-31 OSX 981 911 70 report raw data
                                                                                            
        
2017-05-30 OSX 981 978 3 report raw data
                                                                                                   
 
2017-05-26 OSX 981 911 70 report raw data
                                                                                            
        
2017-05-25 OSX 981 978 3 report raw data
                                                                                                   
 
2017-05-21 OSX 981 911 70 report raw data
                                                                                            
        
2017-05-20 OSX 981 978 3 report raw data
                                                                                                   
 
2017-05-08 OSX 981 911 70 report raw data
                                                                                            
        
2017-05-07 OSX 981 978 3 report raw data
                                                                                                   
 
2017-05-06 OSX 981 981 0 report raw data
                                                                                                    
2017-05-03 OSX 981 914 67 report raw data
                                                                                             
       
2017-05-02 OSX 981 981 0 report raw data
                                                                                                    
2017-05-02 OSX 981 914 67 report raw data
                                                                                             
       
2017-05-01 OSX 981 981 0 report raw data
                                                                                                    
2017-05-01 Linux 981 981 0 report raw data
                                                                                                    
2017-04-28 OSX 981 914 67 report raw data
                                                                                             
       
2017-04-16 Linux 981 975 6 report raw data
                                                                                                   
 
2017-04-15 Linux 981 981 0 report raw data
                                                                                                    
2017-02-09 Linux 947 947 0 report raw data
                                                                                                    
2016-09-26 Linux 947 947 0 report raw data
                                                                                                    
2016-09-25 Windows 947 928 19 report raw data
                                                                                                 
   
2016-09-25 Linux 947 947 0 report raw data
                                                                                                    
2016-09-24 Windows 947 930 17 report raw data
                                                                                                  
  
2016-09-23 Windows 947 928 19 report raw data
                                                                                                 
   
2016-09-08 Windows 947 930 17 report raw data
                                                                                                  
  
2016-09-07 Windows 947 928 19 report raw data
                                                                                                 
   
2016-09-02 Windows 947 930 17 report raw data
                                                                                                  
  
2016-09-01 Windows 947 928 19 report raw data
                                                                                                 
   
2016-08-27 OSX 947 947 0 report raw data
                                                                                                    
2016-08-25 Windows 947 930 17 report raw data
                                                                                                  
  
2016-08-24 Windows 947 928 19 report raw data
                                                                                                 
   
2016-08-16 Windows 947 930 17 report raw data
                                                                                                  
  
2016-08-15 Windows 947 928 19 report raw data
                                                                                                 
   
2016-08-14 Windows 947 930 17 report raw data
                                                                                                  
  
2016-08-13 Windows 947 928 19 report raw data
                                                                                                 
   
2016-08-10 Windows 947 930 17 report raw data
                                                                                                  
  
2016-08-09 Windows 947 928 19 report raw data
                                                                                                 
   
2016-08-05 Windows 947 930 17 report raw data
                                                                                                  
  
2016-08-04 Windows 947 928 19 report raw data
                                                                                                 
   
2016-07-31 Windows 947 930 17 report raw data
                                                                                                  
  
2016-07-30 Windows 947 928 19 report raw data
                                                                                                 
   
2016-07-29 Windows 947 930 17 report raw data
                                                                                                  
  
2016-07-28 Windows 947 928 19 report raw data
                                                                                                 
   
2016-07-19 Windows 947 930 17 report raw data
                                                                                                  
  
2016-07-18 Windows 947 928 19 report raw data
                                                                                                 
   
2016-07-15 Windows 947 929 18 report raw data
                                                                                                  
  
2016-07-14 Windows 947 928 19 report raw data
                                                                                                 
   
2016-07-13 OSX 947 880 67 report raw data
                                                                                            
        
2016-07-13 Windows 947 863 84 report raw data
                                                                                           
         
2016-07-13 Linux 947 813 134 report raw data
                                                                                     
               
2016-07-12 OSX 947 947 0 report raw data
                                                                                                    
2016-07-12 Windows 947 928 19 report raw data
                                                                                                 
   
2016-07-12 Linux 947 947 0 report raw data
                                                                                                    
2016-07-05 Windows 947 930 17 report raw data
                                                                                                  
  
2016-05-14 Windows 947 928 19 report raw data
                                                                                                 
   
2016-05-13 Windows 972 934 38 report raw data
                                                                                                
    
2016-05-13 Linux 972 972 0 report raw data
                                                                                                    
2016-05-12 OSX 972 972 0 report raw data
                                                                                                    
2016-04-13 Windows 972 933 39 report raw data
                                                                                               
     
2016-04-12 Windows 972 934 38 report raw data
                                                                                                
    
2016-03-11 Linux 972 970 2 report raw data
                                                                                                   
 
2016-02-20 OSX 972 905 67 report raw data
                                                                                             
       
2016-02-20 Linux 972 836 136 report raw data
                                                                                      
              
2016-02-19 OSX 972 972 0 report raw data
                                                                                                    
2016-02-19 Linux 972 970 2 report raw data
                                                                                                   
 
2016-02-13 Linux 972 970 2 report raw data
                                                                                                   
 
2016-02-07 Windows 972 933 39 report raw data
                                                                                               
     
2016-02-04 Windows 972 933 39 report raw data
                                                                                               
     
2016-02-04 Linux 972 970 2 report raw data
                                                                                                   
 
2016-01-31 Windows 972 933 39 report raw data
                                                                                               
     
2016-01-31 Linux 972 970 2 report raw data
                                                                                                   
 
2016-01-24 Windows 972 933 39 report raw data
                                                                                               
     
2016-01-24 Linux 972 970 2 report raw data
                                                                                                   
 
2015-12-23 Windows 972 930 42 report raw data
                                                                                               
     
2015-12-23 Linux 972 964 8 report raw data
                                                                                                   
 
2015-12-22 Windows 972 933 39 report raw data
                                                                                               
     
2015-12-22 Linux 972 970 2 report raw data
                                                                                                   
 
2015-12-20 Linux 658 656 2 report raw data
                                                                                                   
 
2015-10-11 Windows 658 619 39 report raw data
                                                                                              
      
2015-10-01 Windows 658 623 35 report raw data
                                                                                              
      
2015-09-24 OSX 646 646 0 report raw data
                                                                                                    
2015-09-24 Windows 646 611 35 report raw data
                                                                                              
      
2015-09-22 Linux 646 644 2 report raw data
                                                                                                   
 
2015-09-16 OSX 644 639 5 report raw data
                                                                                                   
 
2015-09-16 Windows 644 605 39 report raw data
                                                                                             
       
2015-09-16 Linux 644 638 6 report raw data
                                                                                                   
 
2015-09-13 Linux 628 608 20 report raw data
                                                                                                
    
2015-07-01 Linux 544 524 20 report raw data
                                                                                                
    
2015-06-23 Linux 544 526 18 report raw data
                                                                                                
    
2015-04-09 Windows 544 508 36 report raw data
                                                                                             
       
2015-04-09 Linux 544 524 20 report raw data
                                                                                                
    
2015-03-29 Windows 542 507 35 report raw data
                                                                                             
       
2015-03-29 Linux 542 522 20 report raw data
                                                                                                
    
2015-03-22 OSX 542 522 20 report raw data
                                                                                                
    
2015-01-18 Windows 542 510 32 report raw data
                                                                                              
      
2014-12-27 OSX 500 498 2 report raw data
                                                                                                   
 
2014-12-27 Windows 500 498 2 report raw data
                                                                                                   
 
2014-12-27 Linux 500 498 2 report raw data
                                                                                                   
 
2014-08-02 Linux 500 496 4 report raw data
                                                                                                   
 
2014-02-23 Windows 498 496 2 report raw data
                                                                                                   
 
2014-02-23 Linux 498 495 3 report raw data
                                                                                                   
 
2014-02-22 Linux 498 496 2 report raw data
                                                                                                   
 
2014-02-20 OSX 498 496 2 report raw data
                                                                                                   
 
2014-01-24 Linux 498 498 0 report raw data
                                                                                                    
2014-01-08 OSX 496 494 2 report raw data
                                                                                                   
 
2014-01-08 Linux 496 496 0 report raw data
                                                                                                    
2014-01-06 Linux 496 495 1 report raw data
                                                                                                   
 
2014-01-02 OSX 494 492 2 report raw data
                                                                                                   
 
2014-01-02 Windows 494 494 0 report raw data
                                                                                                    
2014-01-02 Linux 494 494 0 report raw data
                                                                                                    
2013-12-29 Linux 494 494 0 report raw data
                                                                                                    
2013-12-01 OSX 494 492 2 report raw data
                                                                                                   
 
2013-10-09 Windows 494 494 0 report raw data
                                                                                                    
2013-10-09 Linux 494 494 0 report raw data
                                                                                                    
2013-07-06 OSX 490 488 2 report raw data
                                                                                                   
 
2013-07-06 Windows 490 490 0 report raw data
                                                                                                    
2013-07-06 Linux 490 490 0 report raw data
                                                                                                    
2013-05-19 OSX 418 416 2 report raw data
                                                                                                   
 
2013-05-19 Windows 418 418 0 report raw data
                                                                                                    
2013-05-19 Linux 418 418 0 report raw data
                                                                                                    
2013-04-20 OSX 416 414 2 report raw data
                                                                                                   
 
2013-04-20 Windows 416 416 0 report raw data
                                                                                                    
2013-04-20 Linux 416 416 0 report raw data
                                                                                                    
2013-01-31 OSX 416 411 5 report raw data
                                                                                                  
  
2012-11-30 Windows 416 415 1 report raw data
                                                                                                   
 
2012-11-30 Linux 416 415 1 report raw data
                                                                                                   
 
2012-11-29 OSX 412 407 5 report raw data
                                                                                                  
  
2012-11-24 Windows 414 414 0 report raw data
                                                                                                    
2012-11-24 Linux 414 414 0 report raw data
                                                                                                    
2012-11-12 Windows 412 412 0 report raw data
                                                                                                    
2012-11-12 Linux 412 412 0 report raw data
                                                                                                    
2012-09-09 Windows 412 343 69 report raw data
                                                                                   
                 
2012-08-31 OSX 410 405 5 report raw data
                                                                                                  
  
2012-08-31 Linux 410 410 0 report raw data
                                                                                                    
2012-08-28 OSX 408 403 5 report raw data
                                                                                                  
  
2012-08-28 Windows 408 408 0 report raw data
                                                                                                    
2012-08-28 Linux 408 408 0 report raw data
                                                                                                    
2012-08-17 OSX 406 402 4 report raw data
                                                                                                   
 
2012-08-17 Windows 406 406 0 report raw data
                                                                                                    
2012-08-17 Linux 406 406 0 report raw data
                                                                                                    
2012-07-31 OSX 406 401 5 report raw data
                                                                                                  
  
2012-07-21 Windows 404 404 0 report raw data
                                                                                                    
2012-07-12 Windows 402 402 0 report raw data
                                                                                                    
2012-07-12 Linux 402 402 0 report raw data
                                                                                                    
2012-07-07 Windows 402 402 0 report raw data
                                                                                                    
2012-07-07 Linux 402 402 0 report raw data
                                                                                                    
2012-03-05 Windows 402 402 0 report raw data
                                                                                                    
2012-02-24 OSX 402 392 10 report raw data
                                                                                                 
   
2012-02-19 Windows 392 392 0 report raw data
                                                                                                    
2012-02-19 Linux 392 392 0 report raw data
                                                                                                    
2012-02-15 Windows 388 388 0 report raw data
                                                                                                    
2012-02-15 Linux 388 388 0 report raw data
                                                                                                    
2012-01-08 Linux 388 382 6 report raw data
                                                                                                  
  
2012-01-07 Linux 388 388 0 report raw data
                                                                                                    
2011-12-29 Linux 384 384 0 report raw data
                                                                                                    
2011-10-03 Linux 382 382 0 report raw data
                                                                                                    
2011-08-20 Linux 376 376 0 report raw data
                                                                                                    
2011-08-16 Linux 376 376 0 report raw data
                                                                                                    
2011-06-13 Linux 376 374 2 report raw data
                                                                                                   
 
2011-06-05 Linux 376 355 21 report raw data
                                                                                              
      
2011-06-03 Linux 376 374 2 report raw data
                                                                                                   
 
2011-05-25 Linux 376 374 2 report raw data
                                                                                                   
 
2011-04-27 Linux 374 372 2 report raw data
                                                                                                   
 
2011-02-18 Linux 374 372 2 report raw data
                                                                                                   
 
2011-02-01 Linux 374 371 3 report raw data
                                                                                                   
 
2011-01-30 Linux 348 346 2 report raw data
                                                                                                   
 
2011-01-18 Linux 348 348 0 report raw data
                                                                                                    
2011-01-11 Linux 346 346 0 report raw data
                                                                                                    
2010-12-19 Linux 344 344 0 report raw data
                                                                                                    
2010-12-14 Linux 340 340 0 report raw data
                                                                                                    
2010-10-08 Linux 340 340 0 report raw data
                                                                                                    
2010-10-02 Linux 340 339 1 report raw data
                                                                                                   
 
2010-10-02 Linux 340 339 1 report raw data
                                                                                                   
 
2010-09-26 Linux 334 327 7 report raw data
                                                                                                 
   
2010-09-25 Linux 334 333 1 report raw data
                                                                                                   
 
2010-09-20 Linux 320 319 1 report raw data
                                                                                                   
 
2010-09-19 Linux 316 314 2 report raw data
                                                                                                   
 
2010-09-18 Linux 312 310 2 report raw data
                                                                                                   
 
2010-09-14 Linux 308 306 2 report raw data
                                                                                                   
 
2010-09-13 Linux 308 306 2 report raw data
                                                                                                   
 
2010-09-12 Linux 296 294 2 report raw data
                                                                                                   
 
2010-09-09 Linux 296 293 3 report raw data
                                                                                                  
  
2010-09-04 Linux 280 277 3 report raw data
                                                                                                  
  
2010-09-03 Linux 270 267 3 report raw data
                                                                                                  
  
2010-09-02 Linux 268 266 2 report raw data
                                                                                                   
 
2010-09-01 Linux 298 277 21 report raw data
                                                                                            
        
2010-08-31 Linux 298 276 22 report raw data
                                                                                            
        
2010-08-30 Linux 298 271 27 report raw data
                                                                                          
          
Windows,2017-08-17,981,969,12 OSX,2017-08-17,981,978,3 Linux,2017-08-17,981,981,0