2024/05/10

Friday, May 10, 2024 (#131) Woozle's Journal

Thursday Friday Saturday

Exact day: category (1) This month: category (6) / page Other years: category (3) This year: category (6) / page

Discuss: APub fedi (Mastodon)

Non-Technical Summary Gmail is blocking messages from my email server. It turns out the address was somehow retroactively placed on 3 blocklists, from an incident 3+ years go, before I was using it. I've managed to get off 2 of them. The 3rd one is more tricky -- it's for a large block of addresses, and the only ways to get off are (1) get the provider (Digital Ocean, in this case) to do it, (2) pay a not-terrible amount of money to a service called Whitelisted, (3) move to a server that isn't in a blocklist zone. I'm considering which combination of 2 and 3 to go with, because fat chance of #1 happening. I have found a server-center that isn't blocked -- it's in Finland :D

Where It Starts

Two days ago (May 8), I noticed that Gmail is now outright blocking my emails.

<woozalia@gmail.com>: host gmail-smtp-in.l.google.com[172.253.115.27] said:
   550-5.7.1 [68.183.140.54      18] Gmail has detected that this message is
   likely 550-5.7.1 suspicious due to the very low reputation of the sending
   IP address. 550-5.7.1 To best protect our users from spam, the message has
   been blocked. 550-5.7.1 For more information, go to 550 5.7.1
   https://support.google.com/mail/answer/188131

I was rather frightfully put out by this, having already spent many hours/days configuring SPF and DKIM and I don't know what else in order to prevent being spamcanned -- and the message seemed to suggest that there was no remedy or appeals process.

It had been working fine as recently as May 5, so whatever changed was obviously very recent.

cloud1 (which uses the blocked IP address) first spun up on 2020/10/21.

Long story short:

• Actually, no, it happened in August of 2020 -- definitely not sometime between May 5 and May 8, not recent, and in fact over a month before I started using that IP address.
• There are 3 IP blocklists which might be causing this. I've managed to get off two of them; the third involves more effort and possibly some money.

Long Story Not Short

Upon following the "more information" link, I found information which suggested that Gmail was blocking me because I'd ended up on a blocklist somehow, and that this had happened because I was running an open SMTP relay on that server (which is something you just don't do, in this era, because spammers will abuse it).

finding the problem

The first thing I did (yesterday afternoon) was check MX Toolbox to make sure I hadn't inadvertently opened a relay (or that my system had been hacked and one had been installed) -- nope, all good:

Next thing was to see if I could find out what blocklists I'm on, and hopefully why.

Note that this list goes on for about 3-4x that long, and all the rest are green/OK.

getting off the lists

List the First

The next step, then, is how to appeal these listings -- especially given that they're all based on an incident which happened before I was assigned that IP address, and which was in fact over 3 years ago.

SORBS seemed to be telling me that I basically can't right now because they're dealing with a DDoS attack... except that "Re-testing is currently ENABLED", so...?

I was able to create an account and at that point it seemed willing to let me start into the de-listing process -- at which point I first found out just how far back they were reaching:

In any case, I was able to request a de-listing -- which succeeded, and somehow ended me up at Spamhaus (not SORBS), so maybe that "can't do it now [yes we can]" warning was about something else?

List the Second

So that's one down. It was bedtime, so I started in on the next one the next day, i.e. this morning: "SORBS SPAM" -- huh, SORBS again? Didn't I just do them? Well, apparently not...

I saw the SORBS information here first, but the other screen seems rather more readable.

I tried to submit a removal request from there, but it said this:

...which turned out to mean that I had to be actually accessing the web site from the blocked IP address.

Now, that machine is not a desktop. It doesn't have a desktop installed on it. There's Linux trickery you can use in order to run GUI applications -- such as a web browser -- on a remote machine, but this tends to be slow and also adds more computing burden to the machine (lots of libraries need to be installed, and it may cause additional processes to always be run at startup; I don't actually know), so I try to avoid it -- and web browsers tend to be particularly CPU-hungry.

My first thought was to install a very basic browser, since the SORBS site doesn't seem likely to expect a lot of finicky JavaScript stuff to work right -- but then I remembered the existence of Lynx (web browser), a web browser that runs entirely in a text-terminal... and after some stumbling around trying to figure out how to navigate web-pages and click on things without a mouse, success was attained:

List the Third

And now we come to the final blocklist on the list of listed blocks, "UCEPROTECTL3":

The second screen in particular contained some text which seems to throw some light on the situation:

Further information which seems potentially illuminating: As you should know now: It is not you, it is your complete provider which got UCEPROTECT-Level 3 listed.
Your IP 68.183.140.54 was NOT part of abusive action, but you are the one that has freely chosen your provider. By tolerating or ignoring that your provider doesn't care about abusers you are indirectly also supporting the global spam with your money. Seen from this point of view, you really shouldn't wonder about the consequences.

Therefore we recommend:
Please send a complaint to your provider and request they fix this problem immediatly. Think about this: You pay them so that you can use the Internet without problems;

If they are ignoring your complaint or claiming they can't do anything, you should consider changing your provider. There are currently about 105,000 providers worldwide, but only a few hundred make it to get listed into UCEPROTECT-Level 3.

According to the statistics measured against the mailflow of several national authorities in Germany, Austria and Switzerland, those few providers which often end up in our Level 3 are responsible for 50 - 75% of all global spam, while almost no real mail came from their networks and ranges.

...which seems maybe a tad harsh (especially the first paragraph) -- but reading it in Autistic Non-Implicatory Mode, it actually seems like a pretty legit point.

Not On Any of the Lists!

So apparently my primary recourse is to get DigitalOcean to get their house in order. Do I have any confidence that they can or will? Kind of not. However, there are at least two other options:

1. Find a servlet (VPS) host whose IP range is not blocked for bad behavior.
2. Get on WhiteListed:

This, unfortunately, costs money (although the price goes down steeply the further ahead you pay). (Note: CHF - Swiss Francs - are apparently approximately $1.)

I'm thinking I will want to use some combination of the two: pay for a certain amount of WhiteListed time, and also move to another host -- because, of the two available Hetzlets I currently have set up, although one of them (in the US) is on UCE's blocklist, the other one -- in Finland -- is not (see sidebar, because for some reason I wanted to include the entire listing).

Moral of the story: before setting up an email host, check the blocklists for an address that isn't on any.

5/12 followup: unblocked!

Well, Gmail is now accepting my messages again -- so I guess I don't actually need to get off that 3rd list, or at least not so urgently. I'll still be moving my email server to a block that isn't listed there, eventually, but I don't have to rush about it (or pay money to Whitelisted). So that's good.

5/13 followup: MS careth not

<woozalia@outlook.com>: host
    outlook-com.olc.protection.outlook.com[52.101.73.18] said: 550 5.7.1
    Unfortunately, messages from [68.183.140.54] weren't sent. Please contact
    your Internet service provider since part of their network is on our block
    list (S3140). You can also refer your provider to
    http://mail.live.com/mail/troubleshooting.aspx#errors.
    [AMS1EPF0000004A.eurprd04.prod.outlook.com 2024-05-13T14:22:20.493Z
    08DC723349903703] (in reply to MAIL FROM command)

I have to think that maybe the remaining blocklist is the issue here. I need to try emailing myself from Hetz1... but I need to set up a proper server and webmail, first, so I can check off the "Delivery status notification" box (which I'm sure is a header thing that I could do without a webmail client, but then I also need a way to view returned messages and would prefer not to spend an hour figuring out what I'm doing and then getting ambiguous results).