Login Register

Downtime of several hours on two mornings this past week

General discussion about Volvos, Volvo parts, your DIY skills, Volvo ownership, and more. Come on in, introduce yourself and say hi! List Volvo events here. Have a nice Volvo? Show it off here. Do you have a question or comment about how MVS works? Ask here.
Post Reply
User avatar
matthew1  
Site Admin
Posts: 14460
Joined: 14 September 2002
Year and Model: 850 T5, 1997
Location: Denver, Colorado, US
Has thanked: 2650 times
Been thanked: 1240 times
Contact:

Downtime of several hours on two mornings this past week

Post by matthew1 »

On the 16th and 19th MVS server was down for several hours both times. Something caused it external to the server's routine functioning. More on that below.

Screenshot 2024-08-20 at 2.49.17 PM.jpg

Both events maxed the memory and CPU, and those events caused the OS to kill the database each time, which sounds crazy but that's a normal chain of events. I restarted the db both times and things were back to normal.

So what was it?

Memory doesn't spike due to magic, or cosmic alignment. Something causes it. I didn't have to look far:

Screenshot 2024-08-20 at 2.36.07 PM.jpg
Screenshot 2024-08-20 at 2.36.07 PM.jpg (74.29 KiB) Viewed 985 times

Hm. That's a whole lot of users on August 16, the day, hour, and minute the first attack occurred. This strongly points to a DDoS attack. The attack, at ~9400 bots (~30X normal traffic), was weak by industry standards, but MVS is a small server by industry standards, so the effect was server knockout.

Yesterday, after the second event, it was clear this was going to happen again and again so I did a sweeping set of changes (reconfigurations). And lo and behold, as if on cue another attack rolled in... and the server passed. I didn't expect the evening attack, but thanks to whoever cued it up because I got to test my changes. :-D Look at the spike at 18:30... it's just as brutal as the others, but MVS kept ticking, which validated my changes.

Screenshot 2024-08-19 at 11.04.08 PM.jpg

If you want to read more about cyberattacks with very recent data, see this https://radar.cloudflare.com/reports/ddos-2024-q2
Help keep MVS on the web -> click sponsors' links here on MVS when you buy from them.

Also -> Amazon link
. Click that when you go to buy something on Amazon and MVS gets a cut!

1998 V70, no dash lights on

1997 850 T5 [gone] w/ MSD ignition coil, Hallman manual boost controller, injectors, R bumper, OMP strut brace

2004 V70 R [gone]

How to Thank someone for their post

Image

User avatar
Krons
Posts: 1069
Joined: 9 January 2022
Year and Model: 08S60 05XC90 02S60
Location: Des Moines, IA
Has thanked: 193 times
Been thanked: 202 times

Post by Krons »

I thought it was odd when the site wouldn’t pull up. Clearly it’s nefarious groups who hate DIY repair of Volvos.
08 S602.5T/05 XC902.5T/02 S602.4T
08 C702.5T (sold)
05 S402.4i (RIP, timing belt failure)
The non-Swedes:
25 Mazda MX-5 / 17 Frontier Pro-4X / 17 Ford Focus
17 R1200GS / 15 Versys 1000 / 11 DR-Z400S / 07 R1200GSA

User avatar
volvolugnut  
Posts: 6223
Joined: 19 January 2014
Year and Model: 2001 V70
Location: Oklahoma USA
Has thanked: 927 times
Been thanked: 999 times

Post by volvolugnut »

THEY are just jealous of our Volvos and the MVS community.
Matt, do you have a source for server advice like MVS for Volvo owners?
volvolugnut
The Fleet:
Volvo: 2001 V70 T5, 1986 244DL, 1983 245DL, 1975 245DL, 1959 PV544, multiple Volvo parts cars.
Mercedes: 2001 E320, 1973 280, 1974 280C, 1989 300E, 1988 300TE, 1979 300TD, parts cars.
2009 Smart Passion
Ford: 1977 F350, 1964 F150 (2), 1938 Tudor Sedan
Farmall tractors: 1956 400 Diesel, 1946 A
And others.

User avatar
hiroku
Posts: 46
Joined: 28 July 2014
Year and Model: 2002 V70 T5
Location: Lisbon
Has thanked: 21 times
Been thanked: 9 times

Post by hiroku »

Wonder what's so special about Volvo forums that make them a desirable target for these things.

The local Portuguese Volvo forums are currently a barren wasteland. The last 6 months or so are only posts about bots talking to each other about good-looking females in their area.

The UK Volvo Owner's Club was apparently also attacked repeatedly by Portuguese IPs, which is why I am blocked from accessing it from my home Wi-Fi (I am innocent though, I swear!).

User avatar
matthew1  
Site Admin
Posts: 14460
Joined: 14 September 2002
Year and Model: 850 T5, 1997
Location: Denver, Colorado, US
Has thanked: 2650 times
Been thanked: 1240 times
Contact:

Post by matthew1 »

hiroku wrote: 21 Aug 2024, 06:27 Wonder what's so special about Volvo forums that make them a desirable target for these things.
This chart is particularly chilling to me. What if it is a competitor? How could one forum operator do this to another? I'm not pointing the finger, but data is data.

Screenshot 2024-08-21 at 7.18.02 AM.jpg

That's taken from the link I included at the bottom of my head post.

I read many years ago a phrase that put the Internet into a perspective that's served me well over the years: the Internet is a dark, mysterious place (paraphrased).

MVS has a wide footprint and a longtime (2001, domain name registered 2002) persistence that gives it a certain profile on the web that will pop up on Bad Actor's target lists. That's just the nature of the beast.

So who knows.
Help keep MVS on the web -> click sponsors' links here on MVS when you buy from them.

Also -> Amazon link
. Click that when you go to buy something on Amazon and MVS gets a cut!

1998 V70, no dash lights on

1997 850 T5 [gone] w/ MSD ignition coil, Hallman manual boost controller, injectors, R bumper, OMP strut brace

2004 V70 R [gone]

How to Thank someone for their post

Image

User avatar
BlackBart
Posts: 6492
Joined: 10 December 2016
Year and Model: 2004 XC70 BlackBetty
Location: Over the far far mountains
Has thanked: 927 times
Been thanked: 884 times

Post by BlackBart »

DDoS = Dedicated Division of Sabotage??
ex-1984 245T wagon
1994 850T5 wagon
2004 XC70 wagon BlackBetty

User avatar
matthew1  
Site Admin
Posts: 14460
Joined: 14 September 2002
Year and Model: 850 T5, 1997
Location: Denver, Colorado, US
Has thanked: 2650 times
Been thanked: 1240 times
Contact:

Post by matthew1 »

Help keep MVS on the web -> click sponsors' links here on MVS when you buy from them.

Also -> Amazon link
. Click that when you go to buy something on Amazon and MVS gets a cut!

1998 V70, no dash lights on

1997 850 T5 [gone] w/ MSD ignition coil, Hallman manual boost controller, injectors, R bumper, OMP strut brace

2004 V70 R [gone]

How to Thank someone for their post

Image

User avatar
abscate
MVS Moderator
Posts: 35267
Joined: 17 February 2013
Year and Model: 99: V70s S70s,05 V70
Location: Port Jefferson Long Island NY
Has thanked: 1497 times
Been thanked: 3810 times

Post by abscate »

Denial of Service. Basically a traffic jam.
Empty Nester
A Captain in a Sea of Estrogen
1999-V70-T5M56 2005-V70-M56 1999-S70 VW T4 XC90-in-Red
Link to Maintenance record thread

User avatar
matthew1  
Site Admin
Posts: 14460
Joined: 14 September 2002
Year and Model: 850 T5, 1997
Location: Denver, Colorado, US
Has thanked: 2650 times
Been thanked: 1240 times
Contact:

Post by matthew1 »

volvolugnut wrote: 20 Aug 2024, 16:15 THEY are just jealous of our Volvos and the MVS community.
Matt, do you have a source for server advice like MVS for Volvo owners?
volvolugnut
Probably 3/4 time I wind up on https://serverfault.com or one of its sister sites to find answers. https://stackoverflow.com https://superuser.com https://askubuntu.com https://dba.stackexchange.com

My 53-year-old body enjoys my 53-year-old brain enjoying solving problems in the nonphysical world of servers rather than the physical world of cars. Both of these things are identical in concept: problem -> find root cause -> learn how to solve root cause -> solve root cause -> test.
Help keep MVS on the web -> click sponsors' links here on MVS when you buy from them.

Also -> Amazon link
. Click that when you go to buy something on Amazon and MVS gets a cut!

1998 V70, no dash lights on

1997 850 T5 [gone] w/ MSD ignition coil, Hallman manual boost controller, injectors, R bumper, OMP strut brace

2004 V70 R [gone]

How to Thank someone for their post

Image

User avatar
matthew1  
Site Admin
Posts: 14460
Joined: 14 September 2002
Year and Model: 850 T5, 1997
Location: Denver, Colorado, US
Has thanked: 2650 times
Been thanked: 1240 times
Contact:

Post by matthew1 »

Three mornings ago we had another attack, largest yet. The site was probably partly or fully inaccessible -- I don't know I was sleeping -- during this time. The database did not crash, instead the MVS server did the best it could during this ~2 hour timeframe and resumed nominal operation when the attack exhausted itself.

I modified protection measures after this.

I'll reset Most Users Ever now so I can get a high water mark again if another attack rolls in.

Screenshot 2024-08-31 at 1.41.43 PM.jpg
Screenshot 2024-08-31 at 1.41.43 PM.jpg (35.35 KiB) Viewed 897 times
Help keep MVS on the web -> click sponsors' links here on MVS when you buy from them.

Also -> Amazon link
. Click that when you go to buy something on Amazon and MVS gets a cut!

1998 V70, no dash lights on

1997 850 T5 [gone] w/ MSD ignition coil, Hallman manual boost controller, injectors, R bumper, OMP strut brace

2004 V70 R [gone]

How to Thank someone for their post

Image

Post Reply