You are the weak(est).link

Last week Craig mentioned a website I've built recently - weak.link. I figured I should probably talk a little about what the site is, why we think it's needed, and how to use it. I'll also geek out a little about how it's built, for those who may be interested.

Firstly, the problem this site is attempting to solve is one that we've seen when sharing URLs in places like the NZ Skeptics Facebook groups, where URLs are flagged as containing misinformation.

The Mailchimp incident

This newsletter might be a good example of this issue - our Mailchimp account was nearly deleted four weeks ago, when a newsletter I sent out was detected by one of Mailchimp's scanners called Omnivore. When I tried to send out the newsletter I received this email:

Your Mailchimp account with the username NZ Skeptics (user ID: XXXXXXXX) has been suspended for violating our Standard Terms of Use and Acceptable Use Policy. Please review our policies for more information on the types of content we don't allow (mailchimp.com/legal/terms).

Eek, I thought, and quickly sent them a response:

_Hi,

I'm guessing there's been a false positive from Omnivore in this case.

We're a Skeptics organisation, and part of what we do is talk critically about bad ideas. Because of this our weekly newsletter is likely to mention ideas that your abuse prevention system will be triggered by, but we're criticising those ideas not promoting them!

It would be great if we could have our account reinstated.

Thanks,

Secretary, NZ Skeptics_

The next email I received from Mailchimp, a couple of hours later, suggested that maybe nobody had bothered to read my email:

_Our automated abuse-prevention system, Omnivore, detected account content or actions that violate our Acceptable Use Policy.

https://mailchimp.com/legal/acceptable_use/#Prohibited_Content

In order to protect all of our users and ensure the deliverability of everyone's campaigns, we have to ask that you seek a new vendor for your marketing needs._

Thankfully Craig was on the case - he jumped onto Twitter, and tweeted at Mailchimp from our account:

@Mailchimp @MailchimpAbuse we're a little disappointed our newsletter has been caught up in your Omnivore system. We fight against bad ideas, not promote them. Can you help us out?

Sure enough, the public shaming was enough for Mailchimp to very quickly reverse their decision, and I was able to send the newsletter out, albeit a couple of hours late.

So, back to the weak.link website. The aim of the website is to allow URLs to be shared in a way that they won't be flagged by automated systems. Although skeptical and critical groups are not spreading misinformation when they share URLs to cults, MLMs and pseudoscientific articles, most of the automated algorithms that social media companies are using these days aren't smart enough to understand this distinction. A bad URL being shared is a bad thing, full stop.

The weak.link website effectively acts as a proxy, allowing people to share links to websites indirectly. It does this by encoding the original URL in the path part of a URL for the weak.link domain (the part after the domain name). So, for example, a link to misinformation.com might end up looking like one of the following URLs:

In the first example, the original URL has been broken up with an underscore (_) to try to obscure it from URL scanners. The second example is a little more hardcore, and involves a custom function I've written to convert the URL between two character sets - the first is the set of characters that are allowed anywhere in a URL, and the second is the set of characters allowed in a URL's path (the bit after the domain name). This conversion efficiently obfuscates the original URL, while ensuring the resulting URL it creates is valid.

For the geeks out there, the data is converted from an 85 character dictionary to a 66 character one, essentially converting the URL from base 85 to base 66. We're essentially converting between these two character sets:

  • ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-._~:/?#[]@!$&'()*+,;%=
  • ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789_.-

The first method is useful if you want to quickly edit a URL by hand, let's say if you have copied a URL into a Facebook post and need to make sure it doesn't get flagged. All you need to do is prefix “weak.link/” to the URL, and then optionally add some illegal domain name characters somewhere in the domain name part of the URL to obscure it further (these characters are pretty much anything other than a-z, 0-9 or a dash (-), so you could use an underscore (_) like the example above, or an asterisk (*), a pipe (|), a hash (#), etc - you get the picture).

This obscuring of the URL appears to be required by Facebook when sharing URLs - my main test subject was Mike Adams' Natural News site, and when I tried to share URLs to his website with the weak.link/ prefix, Facebook would still detect them as bad. But just breaking the domain name up with any text seems to be enough to defeat the algorithm (at least for now). The weak.link site requires for you to break up the URL with an “illegal character”, as it's simple to just remove those from any incoming URLs - if you tried to obscure the domain name with a dash, for example, the site wouldn't know whether to remove that character from the URL or not - it's not psychic!

The more thorough method of obscuring the URL, where it becomes totally unreadable, is the more future proofed of the two. It seems unlikely that social media companies are going to change their detection algorithm to decode my custom encoding when it comes across it.

The front page of the site is intentionally kept simple, and allows you to create and copy a URL using either of the methods described above:

But, what about if some social media company chooses to tell their software to follow the link - won't they see that the original site is on their banned list? In this case no, as the site doesn't automatically redirect you to the original URL - it just gives you a link you can click to visit the site yourself.

This page also displays a clear warning that the site being linked to will contain misinformation, and is only being shared for the purpose of skeptical critique:

Getting Geeky

All links from the site use a “noreferrer” attribute, to stop the misinformation site knowing where the link came from. They also use “nofollow” to ensure that page ranking is not conferred to the site, and they also use “noopener” to prevent access to the window.opener variable (another form of referrer). In essence, this means that the dodgy site doesn't know where the visitor came from when they visit the site, and search engines like google won't consider this link to be positive when it comes to calculating a website's reputation (and therefore its position in search rankings).

Because the original URLs are all encoded in the new weak.link URLs, there's no need for any code to be run on a server, or for a database to save a list of URL translations. This means that the site is a very simple static website, consisting of an HTML file, a CSS file (to make it look nice), and some Javascript code (to get users' browsers to convert the URLs). The site uses Petite Vue, which is a cut-down version of a reactive frontend library called Vue, and Bootstrap. Petite Vue manages frontend updates, such as the URL obscuration and encoding/decoding, and page redirects, and Bootstrap makes it easier to make the website look pretty.

Many years ago there was a similar website to this one called Do Not Link, but it was more complex because of the need for server side code execution and a database of linked URLs. Eventually the creator of that site let it lapse, and although myself and others emailed him to ask if we could take over the domain name and code (as it was a useful skeptical resource), I received no response from him, and all the URLs that used the system became defunct.

My hope is that because my version of this technology is a lot simpler, requiring fewer moving parts, it will keep working as long as we as a Society keep paying for the domain name. The site is hosted on GitHub Pages, which is a free resource for hosting static sites - so there are no hosting costs. So - shameless plug - to help me keep this site up and running, please consider paying to become a member of our society. Your membership fees can help us to pay in advance for the domain name for the next few years, and make sure the site doesn't suffer the same fate as Do Not Link.

If all of this is a little confusing, don't worry! Just go ahead and play around with the site, and see what it does. You won't break anything, as there's nothing there to break. And if you find a bug, just email me (at mark@honeychurch.org) and I'll try to fix it as quickly as I can.