Help mindat.org|Log In|Register|
Home PageMindat NewsThe Mindat ManualHistory of MindatCopyright StatusManagement TeamContact UsAdvertise on Mindat
Donate to MindatSponsor a PageSponsored PagesTop Available PagesMindat AdvertisersAdvertise on MindatThe Mindat Store
Minerals by PropertiesMinerals by ChemistryRandom MineralSearch by minIDLocalities Near MeSearch GlossaryMore Search Options
Search For:
Mineral Name:
Locality Name:
Keyword(s):
 
The Mindat ManualAdd a New PhotoRate PhotosLocality Edit ReportCoordinate Completion ReportAdd Glossary Item
StatisticsThe ElementsMember ListBooks & MagazinesMineral Shows & EventsThe Mindat DirectoryHow to Link to MindatDevice Settings
Photo SearchPhoto GalleriesNew Photos TodayNew Photos YesterdayMembers' Photo GalleriesPast Photo of the Day Gallery
bannerbannerbannerbannerbannerbanner

Bad performance today - Please blame Statoil!

avatar
Jolyon & Katya Ralph September 22, 2011 08:59PM
Hi everyone.

The server wasn't great today - with quite a bit of downtime. On investigating, I found the server was being tied up by a very poorly executed script run by someone at Statoil (statoil.com) which was trying to download all our mineral pages in quick succession.

Here's a very small segment from the log:

ibv2.statoil.com - - <22/Sep/2011:20:33:22 +0000> "GET /min-16611.html HTTP/1.1" 200 22024 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:33:27 +0000> "GET /min-25531.html HTTP/1.1" 200 22191 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:33:32 +0000> "GET /min-25536.html HTTP/1.1" 200 22916 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:33:38 +0000> "GET /min-25526.html HTTP/1.1" 200 22588 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:33:43 +0000> "GET /min-25533.html HTTP/1.1" 200 22307 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:33:48 +0000> "GET /min-25530.html HTTP/1.1" 200 22129 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:33:53 +0000> "GET /min-25534.html HTTP/1.1" 200 22662 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:33:58 +0000> "GET /min-2860.html HTTP/1.1" 200 70600 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:34:03 +0000> "GET /min-25538.html HTTP/1.1" 200 23405 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:34:08 +0000> "GET /min-16610.html HTTP/1.1" 200 22014 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:34:13 +0000> "GET /min-20171.html HTTP/1.1" 200 22184 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:34:18 +0000> "GET /min-20170.html HTTP/1.1" 200 22176 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:34:23 +0000> "GET /min-25537.html HTTP/1.1" 200 22116 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:34:28 +0000> "GET /min-25535.html HTTP/1.1" 200 22041 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:34:33 +0000> "GET /min-27028.html HTTP/1.1" 200 22003 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:34:38 +0000> "GET /min-40144.html HTTP/1.1" 200 48002 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:34:43 +0000> "GET /min-5304.html HTTP/1.1" 200 22175 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:34:48 +0000> "GET /min-2861.html HTTP/1.1" 200 107805 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:34:53 +0000> "GET /min-31206.html HTTP/1.1" 200 22121 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:34:58 +0000> "GET /min-30733.html HTTP/1.1" 200 22697 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:35:03 +0000> "GET /min-30678.html HTTP/1.1" 200 22961 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:35:08 +0000> "GET /min-16612.html HTTP/1.1" 200 22147 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:35:13 +0000> "GET /min-40350.html HTTP/1.1" 200 25908 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:35:18 +0000> "GET /min-30677.html HTTP/1.1" 200 22961 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:35:23 +0000> "GET /min-3695.html HTTP/1.1" 200 70189 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:35:28 +0000> "GET /min-5901.html HTTP/1.1" 200 22079 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:35:33 +0000> "GET /min-2864.html HTTP/1.1" 200 54065 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:35:38 +0000> "GET /min-16614.html HTTP/1.1" 200 22044 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:35:43 +0000> "GET /min-2862.html HTTP/1.1" 200 53391 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:35:48 +0000> "GET /min-16615.html HTTP/1.1" 200 22101 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:35:53 +0000> "GET /min-3694.html HTTP/1.1" 200 77867 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:35:58 +0000> "GET /min-30405.html HTTP/1.1" 200 22973 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:36:03 +0000> "GET /min-16616.html HTTP/1.1" 200 22127 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:36:08 +0000> "GET /min-2863.html HTTP/1.1" 200 65462 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:36:13 +0000> "GET /min-31910.html HTTP/1.1" 200 18577 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:36:34 +0000> "GET /min-31669.html HTTP/1.1" 200 18577 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:36:24 +0000> "GET /min-29814.html HTTP/1.1" 200 18577 "-" "MSIE 5.0"
ibv2.statoil.com - - <22/Sep/2011:20:36:18 +0000> "GET /min-16617.html HTTP/1.1" 200 18577 "-" "MSIE 5.0"

Statoil are now banned from mindat.org so hopefully the problem will go away. Sorry for the inconvenience.

Jolyon
avatar
Peter Andresen September 22, 2011 09:25PM
I promise, I shal not fill gasoline at statoil tomorrow or next time!
avatar
Jolyon & Katya Ralph September 22, 2011 09:35PM
This wasn't the ONLY problem, but it was a contributing factor.

Jolyon
avatar
Robert Knox September 22, 2011 11:44PM
Always nice to know that it's not some malware problem on our computers.
avatar
Paul Brandes September 22, 2011 11:44PM
Statoil????? That's interesting......
avatar
Scott L. Ritchie September 23, 2011 02:48AM
Let's see here, Statoil gives the University of Texas five million dollars to further education, yet allow their IT gang to pilfer data from mindat on the sly... and they won't purchase a single advert here or make a simple donation to help keep the server up and running?

Scott
avatar
Rock Currier September 23, 2011 05:23AM
Why don't you send them, or rather the president of the company an email and ask him why they were trying to hack data from mindat and inform him that the effort nearly crashed the site and invite him to send some money our way.

Rock Currier
Crystals not pistols.
avatar
Jolyon & Katya Ralph September 23, 2011 07:34AM
I have already passed on the information to the network abuse team at Statoil and it's now up to them to investigate and take action internally. Their own firewall logs should be able to identify where this came from within their organization.
avatar
Marcus Voigt September 23, 2011 09:39AM
There are probably many other companies .........

so much valuable information !

Why make a donation to Mindat, if the information can be stolen?

In the end, nobody carries the responsibility for it.

Homo hominis Lupus
avatar
David Bernstein September 23, 2011 10:45AM
Isn't that "homini" Marcus? Regardless, a fine expression to use.
avatar
Van King September 23, 2011 11:46AM
Why not send them a bill? Disruption of a day's service should be worth 5000 Euros.

Best Wishes, Van King
avatar
Alf Olav Larsen September 23, 2011 06:21PM
I am a Statoil employee, and noticed this morning that access to Mindat was forbidden through my job computer. Now I understand why. I am terribly sorry that someone in the company has hacked the site. Hopefully it will never happen again.

Regards,
Alf Olav Larsen
avatar
Jolyon & Katya Ralph September 23, 2011 10:40PM
Alf, very sorry you were inconvenienced by this problem - I'm waiting to hear back from Statoil's network people - once they have assured me the problem has been dealt with then access to the statoil network will be restored.

Jolyon
avatar
Lloyd Llewellyn September 30, 2011 04:28PM
It's very surprising to me that anyone at Statoil has any time to do this - they should normally all be too busy following procedures and management systems! ;)

Lloyd

Norfolk, England
avatar
Debbie Woolf September 30, 2011 11:44PM
Is this the reason why I'm finding mindat slow to respond these past few nights ?
avatar
Douglas Merson October 01, 2011 01:48AM
Here in the Pacific Northwest it was extremely slow until 1500 when it died. Back and not as slow as earlier in the day but not as fast as on a normal day.

Doug
avatar
Corie Mattar October 01, 2011 04:28AM
Someone actually wrote this script to do exactly what it did.
And they're using IE 5.0??
Where can I send my resume to these people?
Obviously they need someone in their tech department from this century! >:D<

Corie
avatar
Jolyon & Katya Ralph October 01, 2011 08:15AM
the IE5.0 is a fake string designed to hide they are using a robot to grab stuff. Just not hiding it very well.
avatar
Jolyon & Katya Ralph October 01, 2011 09:55AM
I didn't hear back from Statoil at all.
avatar
Van King October 01, 2011 11:01PM
As Statoil has been formally notified and there is still recurring denial of service due to running scripts, then their failure to respond indicates at least an unconcern if not a more pernicious attitude. I still suggest that a bill should be sent to Statoil for actions now tolerated by their management. Maybe 5,000 Euoros/day is not enough?

Best Wishes, Van King
avatar
Jolyon & Katya Ralph October 02, 2011 09:46AM
The script at statoil stopped running not long after the ban was put in place.

Jolyon
avatar
Spencer Ivan Mather October 02, 2011 02:04PM
I agree with Van King, send Statoil a big bill, that should stop whoever it is!
avatar
David Bernstein October 02, 2011 02:32PM
It may be an obvious point but without substantiation of damages in place behind a bill of any amount, such a "delivery" would be laughed at. Just sayin' in the event Van's idea gains traction.
avatar
Daniel Levesque October 02, 2011 10:35PM
I'd hope the appropriate measures (ban) would remain in place if there was no response from Statoil's IT staff. Obviously they have an internal issue to deal with:
From "Statoil Ethics- Code of Conduct- version 5"
"4.9 ...Any downloading, storing or disseminating in breach of any copyright law or provision is prohibited."

Sounds like they have a rotten apple in the bunch!!!
avatar
Tony Albini October 02, 2011 11:02PM
Jolyon,

For the past two days, I cannot look at the Today's Mineral Photos. I keep getting a prompt to Register which I have been for a while. Even when I follow the site's prompt, I cannot get in to look at the photos which are so informing. I hope this issue is related to your emails.

Regards,

Tony Albini
avatar
Modris Baum October 03, 2011 12:18AM
Tony - you have to log in as well. According to the Member Directory you last "visited" (i.e. logged in) on Feb 10, 2010)
avatar
Rock Currier October 03, 2011 01:22AM
There are still times when Mindat is very slow, and last night there were times when Google chrome could not locate the website at all. Again!

Rock Currier
Crystals not pistols.
avatar
Debbie Woolf October 03, 2011 10:18AM
I got this message using chrome the other night "Fatal Error - Unable to connect to database."

(:P)
avatar
Van King October 03, 2011 02:28PM
The real purpose to sending a bill to Statoil is not to get money, but to get their attention. A bill sent to the accounting department and to the president should receive some notice. From the various messages, it is not always clear that the Statoil script remains the only problem. Is it a coincidence or is it the same problem?

Best Wishes, Van King
avatar
Jolyon & Katya Ralph October 03, 2011 02:32PM
The statoil script stopped running shortly after I banned them. Statoil are not causing any direct problems to mindat currently
avatar
Rick Dalrymple October 10, 2011 06:35PM
I more often than not have a problem getting on Mindat. I get the "Fatal Error-Unable to Connect to Database" a lot. Sometimes man times a day.

I am also getting the You must login to view these pages" which is ok but then I have to log out to go to a different page and then log in again.

I just thought it was my computer/lines/modem.

Rick
I know I am in my own little world, but everyone knows me here.
avatar
Jolyon & Katya Ralph December 27, 2012 04:24PM
Statoil are back to their bad old tricks and trying again to steal content from mindat with bots.

I have had to block them again.

Jolyon
avatar
Dean Allum December 27, 2012 05:31PM
Jolyon,
This problem with Statoil appears to be clearly in contradiction to their stated policy shown below. The mindat website audience would normally be one of Statoil's staunchest public supporters, so their actions are mystifying.

Statoil Ethics Code of Conduct Version 2:
"4.9 Information and IT systems
The individual’s use of information, IT systems and,in particular, internet services must be based on the
needs of the business and not by personal interests.
Information produced and stored on Statoil’s IT systems is regarded as the Group’s property.
Statoil therefore reserves the right to access all such information except where limited by law or
agreement. The individual is responsible for maintaining electronic files and archives in an orderly manner.
Private use is only permitted to a limited extent. Information that may be considered illegal, offensive
or inappropriate must not be processed, downloaded, stored or disseminated. Any downloading, storing
or disseminating in breach of any copyright law or provision is prohibited.
Any use of software in breach
of any copyright law or provision is prohibited."
avatar
Keith Compton December 27, 2012 11:46PM
Hi J

Perhaps a more permanent ban is required

Cheers
avatar
Jenna Mast December 28, 2012 04:53AM
I guess someone at statoil really wants the entire mindat database for Christmas. I can relate. I often worry "What if someday Mindat should be no more? Then how will I identify my mineral specimens?" May that day never come.
avatar
Chris Mavris December 28, 2012 09:56AM
If they keep acting like this, I would consider a legal action.
avatar
David DeCourcey December 28, 2012 01:26PM
Rather than banning them or taking legal action, ask them to start sponsering a few pages.
avatar
Reiner Mielke December 28, 2012 02:50PM
Hello David,

Why should they sponsor a few pages? You are assuming they have a conscience, which has not been thus far demonstrated.
avatar
Branko Rieck December 28, 2012 04:00PM
Why just a few? I'd say they sponsor every page they've downloaded (that is not already sponsored). That would make life easier for Jolyon...
avatar
David Von Bargen December 28, 2012 04:20PM
Well, it probably isn't anything that Statoil is officially doing. Probably just a bored employee. If I was on their IT staff, I sure wouldn't want someone loading a spider bot on my network.
avatar
Aaron Cross December 31, 2012 07:41PM
Didn't you have a big problem with them earlier??
Spam prevention:
Please, enter the code that you see below in the input field. This is for blocking bots that try to post this form automatically. If the code is hard to read, then just try to guess it right. If you enter the wrong code, a new image is created and you get another chance to enter it right.
CAPTCHA
Your message:
Attachments:
  • Valid attachments: jpg, jpeg, gif, png, pdf
  • No file can be larger than 9.38 MB
  • 3 more file(s) can be attached to this message



bannerbannerbannerbannerbannerbanner
Mineral and/or Locality  
Mindat.org is an outreach project of the Hudson Institute of Mineralogy, a 501(c)(3) not-for-profit organization.
Copyright © mindat.org and the Hudson Institute of Mineralogy 1993-2016, except where stated. Mindat.org relies on the contributions of thousands of members and supporters.
Privacy Policy - Terms & Conditions - Contact Us Current server date and time: December 9, 2016 11:37:02
Go to top of page