Here's what's going on.
Many, or even most of you, run an ad blocker or some other software in attempt to keep your machine safe and secure - or maybe just less annoying. My wife runs Adblock Plus - ABP. Instead of that, I've been running an MVPS hosts file since the early '00s. Let me clip their explanation, for those unfamiliar with the concept.
The Hosts file is loaded into memory (cache) at startup, so there is no need to turn on, adjust or change any settings with the exception of the DNS Client service (see below). Windows automatically looks for the existence of a HOSTS file and if found, checks the HOSTS file first for entries to the web page you just requested. The 0.0.0.0 (prefix) is considered the location of your computer, so when an entry listed in the MVPS HOSTS file is requested on a page you are viewing, your computer thinks 0.0.0.0 is the location of the file. When this file is not located it skips onto the next file and thus the ad server is blocked from loading the banner, Cookie, or some unscrupulous tracker, or javascript file.Like all security, including ad blockers, this is sometimes inconvenient. I get emails regularly that come from companies I have experience and history with, and clicking on their link fires a 404 Not Found error. I have commented out lines in the hosts file for specific companies I know. I'll let the MVPS Hosts folks tell you how to install it and modify it and all, because it depends on what OS you're running. I'm not aware of an equivalent file in Mac OS or any flavor of *nix.
Example - the following entry 0.0.0.0 ad.doubleclick.net blocks all files supplied by that DoubleClick Server to the web page you are viewing. This also prevents the server from tracking your movements. Why? ... because in certain cases "Ad Servers" like Doubleclick (and many others) will try silently to open a separate connection on the webpage you are viewing, record your movements then yes ... follow you to additional sites you may visit.
In the last few weeks, I started to notice that the links that are sent in the daily or weekly newsletters don't work - the pages won't load. It started out with just those news site, but now I find virtually every magazine/trade site I go to won't load.
- Microwaves and RF Magazine - http://www.mwrf.com/
- Electronic Design - http://www.electronicdesign.com/
- Machine Design - http://www.machinedesign.com/
- EETimes - https://www.eetimes.com/
- Design News - https://www.designnews.com/
Lacking their help, I asked the MVPS Hosts file guys if there was software I could run that would tell me what the magazines are doing. We exchanged details including some sites I sent them to try. I got this reply:
They also recommended a free program they use to track these things, Fiddler.After visiting those sites ... what a (coding) mess!Even after renaming the HOSTS file ... the site failed to load properly ... seems I had to turn off my ad blocker too just to get the page to load.Here is the list of the 3rd party trackers/advertisers:If you disable all those ... they are disabled on all sites ... I would recommend renaming the HOSTS file instead ...
Naturally, I commented out all of those sites and tried to load the pages. That didn't work either.
What to do? Disabling the hosts file by renaming it, or having different versions for more safe/less safe browsing, is just that: less safe. On the other hand, I've been reading these magazines for years - decades in most cases - and I tend to trust them. I don't care if they serve me an ad for the latest silicon from Analog Devices or the latest amplifier from Mini Circuits. I don't, however, tend to trust that list of 13 ad servers (or trackers), and I especially don't trust their security. It seems to me that an effective way to target the magazine's readers would be through those ad companies.
I'll keep poking away at it, but that's why the latest post I had about some new technology was February 11th.
I don't think they're "going away" in a classical sense as they're still there; you just can't access them.
ReplyDeleteNo problem here with Firefox and AdBlock plus.
Of all the oddities, the oddest is that on some of those pages, if I happen to position the mouse over an image should be - which I can't see - I can right click, select "view image" and it works perfectly. Still can't see the text, and I've tried highlighting the pages to see if it's all displaying as white on white to no avail.
Delete/etc/hosts in Linux. But host file blocking is garbage. It only prevents DNS resolution. I load stats tracking across and ads from servers via IP address to about hosts file blocking.
ReplyDeleteThat's what I get for typing from my cell phone. I sound like a raving lunatic. ;)
DeleteThe hosts file overrides DNS resolution.
So if my website loads myevil-ad-domain.tld/tracking-code.js and you add '0.0.0.0 myevil-ad-domain.tld' to your hosts file, it forces it to resolve to 0.0.0.0 instead of the 'legitimate' IP.
But what if my website loads tracking code from '209.216.160.1/tracking-code.js'. No DNS resolution required because I used an IP. So I've loaded the very ad or tracking code you think your hosts file is protecting you from.
Hosts file blocking is garbage and can be easily subverted.
Not sounding like a raving lunatic, but I didn't understand all of it.
DeleteI get the part about DNS resolution and the way around that being to load an IP address directly.
What I didn't get was I load stats tracking across and ads from servers via IP address
Yeah--even I have no idea what autocorrect originally butchered.
DeleteIn some cases where I *really* want a script loaded--either for ads or statistics tracking, I load it by IP. You can block domains all you want, but the only way you're preventing that IP from loading is a firewall rule, a routing change, or an add-on in the browser that is looking for that IP or watching what is being loaded.
You may indeed have been "reading these magazines for years - decades in most cases - and (I) tend to trust them", but do you trust them to prevent hackers from posting malware on their website that then invades YOUR computer if you whitelist them?
ReplyDeletehttp://www.straitstimes.com/singapore/slow-browsing-hackers-could-be-mining-bitcoin
Your problems may also lay in how the sites are hosted. They may not be hosted directly from the URL that they have listed - redirected to other servers like Akamai.
ReplyDeleteI've been using the browser Brave for the last year or two and been very happy to not receive the usual amount of junk trackers and such. Much faster browsing with no need for add-ons. CEO is Brendan Eich, the same person who was forced from Mozilla because of his politics (donated against California's same-sex marriage law and was lambasted for such).
Versions for Mac, Windows, Linux.
Wandering Neurons
Somebody mentioned Brave on their blog a while back and I think I even tried it once. Perhaps too early in development and (IIRC) something didn't work.
DeleteI might try it. After my posting that last night, Mrs. Graybeard found that EETimes and the top level Design News were working for her, and she's running the same hosts file as well as Ad Blocker Pro. I found that if I disabled my Avast virus web reputation service, I could load both. Some Design News "features" don't work but I can look at the daily updates. But she loaded them without doing anything to Avast.
It's a convoluted mess.
PaleMoon + uBlock Origin + Microsoft Security Essentials. Seems to be working fine with these websites.
ReplyDeleteOK! I read what you wrote and didn't really understand it. I tried to cut and paste the URL (http://www.machinedesign.com/ ) in and got a blank page. SO I went to Bing and pasted http://www.machinedesign.com/ into the search and came up with a number of links but when I selected one I again got the blank page.
ReplyDeleteSo I see that there is a problem but I do not understand from what you wrote what they problem is or how to fix it. Is it my fault somehow? Something I did setting up my settings? Using the wrong search engine? Using Windows 10? What?
Can you explain it in more detail and not assume we/I know too much about how the Internet works?
What is the HOSTS file? How do I rename it? Why? I never renamed it before, what's different now?
What is a white list? What is an MVPS host?
Well, I thought I covered most of that. As far a I can tell, it's not anything you did. If you haven't deliberately loaded a custom hosts file, I would think Machine Design would load for you. I'm guessing from the question that you haven't loaded somebody's hosts file.
DeleteThe hosts file is a file that Windows loads into memory when you open your browser (so it only loads it once), and uses it as a shortcut to where websites are being hosted. MVPS is an organization whose version of the Hosts file I use, so an MVPS hosts is a file I download from them. It's free.
So let's say go to load some webpage.com. The first thing Windows does is look in the hosts file to see if it knows a shortcut; if not, it goes to your Internet provider's Domain Name Server (DNS) and asks "where is webpage.com?". The DNS gives it the four numbers of the IP address (like 123.456.789.987) and your computer connects to that address. There's a shortcut: if the hosts file says 0.0.0.0, that means "it's here on this computer". If Windows doesn't find and load that in a really short period of time, it skips that and goes to the next address. What the MVPS hosts file does is collect the name of every known adware and malware source and assign it 0.0.0.0. That makes your computer skip trying to load them.
I don't know anything about Windoze 10 because I'm still running Win7. Their website has Windows 10 instructions. On win7, the hosts file is located deep in the Windows directory on your computer, usually C:\Windows\System32\drivers\etc You need to have administrator privileges to rename it in 7; I don't know if that applies to 10.
If you can find your hosts file and look in it, if it has pages and pages of things like
0.0.0.0 www.77tracking.com
0.0.0.0 ak1s.abmr.net
0.0.0.0 targeting.adwebster.com
0.0.0.0 cdn.betrad.com
0.0.0.0 c.betrad.com
0.0.0.0 ads.static.blip.tv
then it's an anti-spyware/anti-adware file from somebody.
Hope that helps.
Re serving ads by IP address, I'm not a network guy, but I'm guessing you could defeat that by modifying your local routing table.
ReplyDeleteAlso, with the push for TLS on websites, that should cause issues for ad severs accessed by IP. You can't get a cert issued to an IP address from a trusted authority AFAIK. So a secure site pulling ads from an insecure site will cause a mixed content error and the insecure content usually isn't displayed.
As always, YMMV.
That's true, but I can think of a handful of ways around that.
DeleteDomain names are cheap. Register a few. Point them to your IP space, keep changing the domain name used to serve the content.
The 'big players' like Google don't change their ad-serving domain name every few minutes--but who cares? It's not Google is serving up infected ads or anything. It's the more 'shady' companies that usually serve up malware or get compromised. And they won't mind changing domain names or even IP space every 24 hours...which makes host-file-based blocking ineffective against them.
Hadn't noticed til you brought it up .. so I checked. I can get EETimes and EDN but not Electronic Design. I have no trouble getting to the prime manufacturers or DigiKey ... and those are >really< important :)
ReplyDeleteQ
Thanks for mentioning EDN. I had forgotten to check them, but it seems to load normally here.
DeleteTried your magazine listing with Win 8.1, Opera 51.0, ABP, Ghostery, Norton 360 Security and got blank pages for all of them.
ReplyDeleteI "trusted" each site with Ghostery and all of them work fine.
Don't think this really helps with the virus stuff but I have my connection via IPVanish, a VPN. Just an FYI.
DeleteI do need to research the VPN world, but haven't yet. You gave me a few things to check there.
DeleteRe: Avast, when I switched from AVG, I read reviews on a few places and it was in the top 2 or 3 everywhere. Can't say I'm particularly happy with any of them.
VPN: I won't make a recommendation - that's an individual decision - but this might be a good site to start a review of providers.
Deletehttps://thatoneprivacysite.net/
Q
"I "trusted" each site with Ghostery and all of them work fine."
DeleteOK! I know there is something there that actually means something. I "trusted" you. Hmmmmmm! OK what does that mean? With "ghostery". As opposed to I don't believe in ghosts? What's a ghostery? Is that where old ghosts go to live? "work fine" I assume this means he saw something besides a blank page but not sure why, has something to do with ghosts or sumpin.
Then there is Opera 51. etc. Is this like area 51? What???
Is it even possible to discuss this in English. It all sounds like geek rap music. Tell me that you do not believe it is possible to explain this so that a normal person who has a life could understand it and I will never ask again. But if it is possible to explain each of the little abbreviations and techy phrases please do so.
I don't want to talk for Q, but I don't know he'll come back to see this.
DeleteOpera is browser and 51 is the version. Like I'm running Firefox 58.0.2 (I had to look that up). Ghostery is a program that tells you what the website is trying to load onto your computer, and trusting it means you've told Ghostery it's OK for the website to load everything onto your computer. ( https://www.ghostery.com/ )
Just checked 2018 listings and I guess things have changed. You might want to look at them again. Avast has gone down in rankings. I am looking because I communicated to Symantec, Norton's owner that due to their punishment of NRA members, I would not be renewing my subscription to their service. So I will be looking for a replacement. Happy Hunting on more than two fronts!
DeleteYou are concerned about your security but are using Avast anti-virus?? I would reconsider my friend. Not very good, in my opinion.
ReplyDeleteDepending on the browser you're running, you may be able to get a plugin called noscript, that lets you kill the scripts that get loaded form those sites on a case by case, or temporary basis.
ReplyDeleteOn the plus side it also gets rid of the 'hey whitelist our page' popups that show up when you block ads.
Thanks! I'll check it out.
Delete