Mindat Logo
bannerbannerbannerbanner
Welcome!

Bad performance today - Please blame Statoil!

avatar Bad performance today - Please blame Statoil!
September 22, 2011 08:59PM
Hi everyone.

The server wasn't great today - with quite a bit of downtime. On investigating, I found the server was being tied up by a very poorly executed script run by someone at Statoil (statoil.com) which was trying to download all our mineral pages in quick succession.

Here's a very small segment from the log:

ibv2.statoil.com - - [22/Sep/2011:20:33:22 +0000] "GET /min-16611.html HTTP/1.1" 200 22024 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:33:27 +0000] "GET /min-25531.html HTTP/1.1" 200 22191 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:33:32 +0000] "GET /min-25536.html HTTP/1.1" 200 22916 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:33:38 +0000] "GET /min-25526.html HTTP/1.1" 200 22588 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:33:43 +0000] "GET /min-25533.html HTTP/1.1" 200 22307 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:33:48 +0000] "GET /min-25530.html HTTP/1.1" 200 22129 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:33:53 +0000] "GET /min-25534.html HTTP/1.1" 200 22662 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:33:58 +0000] "GET /min-2860.html HTTP/1.1" 200 70600 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:34:03 +0000] "GET /min-25538.html HTTP/1.1" 200 23405 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:34:08 +0000] "GET /min-16610.html HTTP/1.1" 200 22014 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:34:13 +0000] "GET /min-20171.html HTTP/1.1" 200 22184 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:34:18 +0000] "GET /min-20170.html HTTP/1.1" 200 22176 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:34:23 +0000] "GET /min-25537.html HTTP/1.1" 200 22116 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:34:28 +0000] "GET /min-25535.html HTTP/1.1" 200 22041 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:34:33 +0000] "GET /min-27028.html HTTP/1.1" 200 22003 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:34:38 +0000] "GET /min-40144.html HTTP/1.1" 200 48002 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:34:43 +0000] "GET /min-5304.html HTTP/1.1" 200 22175 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:34:48 +0000] "GET /min-2861.html HTTP/1.1" 200 107805 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:34:53 +0000] "GET /min-31206.html HTTP/1.1" 200 22121 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:34:58 +0000] "GET /min-30733.html HTTP/1.1" 200 22697 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:35:03 +0000] "GET /min-30678.html HTTP/1.1" 200 22961 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:35:08 +0000] "GET /min-16612.html HTTP/1.1" 200 22147 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:35:13 +0000] "GET /min-40350.html HTTP/1.1" 200 25908 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:35:18 +0000] "GET /min-30677.html HTTP/1.1" 200 22961 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:35:23 +0000] "GET /min-3695.html HTTP/1.1" 200 70189 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:35:28 +0000] "GET /min-5901.html HTTP/1.1" 200 22079 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:35:33 +0000] "GET /min-2864.html HTTP/1.1" 200 54065 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:35:38 +0000] "GET /min-16614.html HTTP/1.1" 200 22044 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:35:43 +0000] "GET /min-2862.html HTTP/1.1" 200 53391 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:35:48 +0000] "GET /min-16615.html HTTP/1.1" 200 22101 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:35:53 +0000] "GET /min-3694.html HTTP/1.1" 200 77867 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:35:58 +0000] "GET /min-30405.html HTTP/1.1" 200 22973 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:36:03 +0000] "GET /min-16616.html HTTP/1.1" 200 22127 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:36:08 +0000] "GET /min-2863.html HTTP/1.1" 200 65462 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:36:13 +0000] "GET /min-31910.html HTTP/1.1" 200 18577 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:36:34 +0000] "GET /min-31669.html HTTP/1.1" 200 18577 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:36:24 +0000] "GET /min-29814.html HTTP/1.1" 200 18577 "-" "MSIE 5.0"
ibv2.statoil.com - - [22/Sep/2011:20:36:18 +0000] "GET /min-16617.html HTTP/1.1" 200 18577 "-" "MSIE 5.0"

Statoil are now banned from mindat.org so hopefully the problem will go away. Sorry for the inconvenience.

Jolyon
avatar Re: Bad performance today - Please blame Statoil!
September 22, 2011 09:25PM
    
I promise, I shal not fill gasoline at statoil tomorrow or next time!
avatar Re: Bad performance today - Please blame Statoil!
September 22, 2011 09:35PM
This wasn't the ONLY problem, but it was a contributing factor.

Jolyon
Re: Bad performance today - Please blame Statoil!
September 22, 2011 11:44PM
    
Always nice to know that it's not some malware problem on our computers.
avatar Re: Bad performance today - Please blame Statoil!
September 22, 2011 11:44PM
Statoil????? That's interesting......
avatar Re: Bad performance today - Please blame Statoil!
September 23, 2011 02:48AM
    
Let's see here, Statoil gives the University of Texas five million dollars to further education, yet allow their IT gang to pilfer data from mindat on the sly... and they won't purchase a single advert here or make a simple donation to help keep the server up and running?

Scott
avatar Re: Bad performance today - Please blame Statoil!
September 23, 2011 05:23AM
Why don't you send them, or rather the president of the company an email and ask him why they were trying to hack data from mindat and inform him that the effort nearly crashed the site and invite him to send some money our way.

Rock Currier
Crystals not pistols.
avatar Re: Bad performance today - Please blame Statoil!
September 23, 2011 07:34AM
I have already passed on the information to the network abuse team at Statoil and it's now up to them to investigate and take action internally. Their own firewall logs should be able to identify where this came from within their organization.
Re: Bad performance today - Please blame Statoil!
September 23, 2011 09:39AM
There are probably many other companies .........

so much valuable information !

Why make a donation to Mindat, if the information can be stolen?

In the end, nobody carries the responsibility for it.

Homo hominis Lupus
avatar Re: Bad performance today - Please blame Statoil!
September 23, 2011 10:45AM
Isn't that "homini" Marcus? Regardless, a fine expression to use.
avatar Re: Bad performance today - Please blame Statoil!
September 23, 2011 11:46AM
    
Why not send them a bill? Disruption of a day's service should be worth 5000 Euros.

Best Wishes, Van King
Re: Bad performance today - Please blame Statoil!
September 23, 2011 06:21PM
I am a Statoil employee, and noticed this morning that access to Mindat was forbidden through my job computer. Now I understand why. I am terribly sorry that someone in the company has hacked the site. Hopefully it will never happen again.

Regards,
Alf Olav Larsen
avatar Re: Bad performance today - Please blame Statoil!
September 23, 2011 10:40PM
Alf, very sorry you were inconvenienced by this problem - I'm waiting to hear back from Statoil's network people - once they have assured me the problem has been dealt with then access to the statoil network will be restored.

Jolyon
avatar Re: Bad performance today - Please blame Statoil!
September 30, 2011 04:28PM
It's very surprising to me that anyone at Statoil has any time to do this - they should normally all be too busy following procedures and management systems! ;)

Lloyd

Norfolk, England
avatar Re: Bad performance today - Please blame Statoil!
September 30, 2011 11:44PM
    
Is this the reason why I'm finding mindat slow to respond these past few nights ?
avatar Re: Bad performance today - Please blame Statoil!
October 01, 2011 01:48AM
    
Here in the Pacific Northwest it was extremely slow until 1500 when it died. Back and not as slow as earlier in the day but not as fast as on a normal day.

Doug
avatar Re: Bad performance today - Please blame Statoil!
October 01, 2011 04:28AM
    
Someone actually wrote this script to do exactly what it did.
And they're using IE 5.0??
Where can I send my resume to these people?
Obviously they need someone in their tech department from this century! >:D<

Corie
avatar Re: Bad performance today - Please blame Statoil!
October 01, 2011 08:15AM
the IE5.0 is a fake string designed to hide they are using a robot to grab stuff. Just not hiding it very well.
avatar Re: Bad performance today - Please blame Statoil!
October 01, 2011 09:55AM
I didn't hear back from Statoil at all.
avatar Re: Bad performance today - Please blame Statoil!
October 01, 2011 11:01PM
    
As Statoil has been formally notified and there is still recurring denial of service due to running scripts, then their failure to respond indicates at least an unconcern if not a more pernicious attitude. I still suggest that a bill should be sent to Statoil for actions now tolerated by their management. Maybe 5,000 Euoros/day is not enough?

Best Wishes, Van King
Author:

Subject:


Attachments:
  • Valid attachments: jpg, gif, png, pdf
  • No file can be larger than 1000 KB
  • 3 more file(s) can be attached to this message

Spam prevention:
Please, enter the code that you see below in the input field. This is for blocking bots that try to post this form automatically. If the code is hard to read, then just try to guess it right. If you enter the wrong code, a new image is created and you get another chance to enter it right.
CAPTCHA
Message:
Mineral and/or Locality  
Search Google  
Copyright © Jolyon Ralph and Ida Chau 1993-2014. Site Map. Locality, mineral & photograph data are the copyright of the individuals who submitted them. Site hosted & developed by Jolyon Ralph. Mindat.org is an online information resource dedicated to providing free mineralogical information to all. Mindat relies on the contributions of thousands of members and supporters. Mindat does not offer minerals for sale. If you would like to add information to improve the quality of our database, then click here to register.
Current server date and time: October 23, 2014 01:37:50
Mineral and Locality Search
Mineral:
and/or Locality:
Options
Fade toolbar when not in focusFix toolbar to bottom of page
Hide Social Media Links
Slideshow frame delay seconds