Searching for Search Engine and Post-Google Dream

15.05.2020

Numbers strongly suggest that for the last several years people have been doing a lot of googling and really not much of binging, yahooing, and even baiduing in comparison. When it comes to certain alternative solutions like DuckDuckGo, there’s even more ground to gain. And there are reasons to want these leftfield solutions gaining ground, privacy being the biggest.

In this piece, we outline the main problems with mainstream search engines and suggest solutions with some important tips on the side.

Mainstream Search Engines and Their Problems

Google, boasting about 93% market share in 2019, is undoubtedly the mainstream search engine. No wonder they get their own verb. Yet, names such as Bing and Yahoo! are still somewhat recognizable. There are also local players like Baidu in China and Yandex in post-soviet countries. All these search engines have quite a lot in common, both good and bad.

Search engines by market share in 2019

Search engines by market share in 2019. Source: Wikipedia, data by Statcounter

At the fundamental level, typical search engines work the same way. They use bots called web crawlers or spiders that automatically surf the World Wide Web and make a machine-readable catalog of web pages. When a user types in a text search query, the software checks back with the catalog and uses various complex methods like natural language processing to find relevant pages to return. Then, proprietary in-house magic happens and the user sees a search engine results page (SERP), hopefully showing the things they’ve been looking for.

Although, with Google and its smaller mainstream competitors the story wouldn’t end with the SERP. The search engine records the query while the visited website gets to know that a certain user clicked that link in SERP. That’s some useful information. This way, websites can see their main sources of users and adjust their strategy to be found more often. And Google can learn more about us, which would help the company provide a better user experience customized to our individual preferences. The thing is that Google, Bing, and the like also sell users’ data to advertisers as their business model.

There are problems with the way modern search engines treat users’ personal information, search result prioritization, and ads.

The first glaring issue is privacy. Google and other mainstream search engines collect information about users’ search queries and preferences. The information is then used to target ads with better precision, suggest autocomplete results, and so on. These companies make money from advertising deals and they are interested in offering better ROI to clients. Advertisers can misuse or leak users’ data and authorities can force the search engine company to share some insights. This isn’t good and can be prevented by not storing or recording such information on the search engine’s side whatsoever.

Another issue is the filter bubbles. Internet users encounter those on social media and in search results. Algorithms that form Facebook news feeds and search results lists on Google are designed to prioritize the results they think the user wants to see. The more these algorithms know about the user, the better they manage to pick posts, links, and ads that are more likely to get the user engaged. They show more and more things you agree with, while the opposing views go down on the list.

When lots of people get their news from the internet believing that they get the full picture, filter bubbles can cause harm. After a while, thanks to algorithms cherry-picking content people will have their views solidified remaining largely ignorant of opposing arguments. This doesn’t help any sort of constructive dialog.

The magic algorithms that form our SERPs on Google will also take into account the interests of advertisers, putting promotions among the results and showing said targeted ads. Meanwhile, there’s an entire profession dedicated solely to making web pages appear higher in SERP. They call it search engine optimization or SEO for short. Some SEO-people will sell you artifacts and charms promised to increase search traffic, others will earn their pay by tweaking your website so it pleases cryptic algorithms that decide what’s relevant. Professionals work day and night to make certain websites appear more often in your search results and you are helping them by letting search engines keep tabs on you.

And then there are targeted ads. You google a store to buy a doormat and the next thing you know there are doormat ads everywhere. That’s just a thing people live with. While mildly annoying, it may lead to potentially harmful consequences. Ads can relate to political views, ethnicity, sexual preferences, and other sensitive things. They can make your roommate suspect that you are secretly looking for another apartment, but they can also become the channel for propaganda.

A Glimpse of Privacy-Focused Alternatives

The way to avoid problems with mainstream search engines is to avoid mainstream search engines. There are multiple alternatives that made it their thing to refrain from logging users’ activity, offer encryption, and generally care for privacy. As there are also numerous top-something lists of privacy-focused search engines that can be found by the means of googling, we will share some of the basics so you can enjoy enhanced privacy immediately.

DuckDuckGo is the one that gets most of the attention when it comes to private search. On the front, it is a fully-capable search engine featuring all the tools and knick-knacks you’d expect. It even has a night mode, which is a much-appreciated feature for those who dwell in the dark.

DuckDuckGo search results page

As pointed out by Wired’s James Temperton, DuckDuckGo is up to most of the day-to-day search tasks, but more complex requests may leave the algorithm guessing. Knowing you well is very helpful in trying to guess what you meant. This search engine also doesn’t have a feature of filtering things like image search results by license type, which is an inconvenience.

Fortunately, there’s a dedicated search engine that lets you search for images distributed under Creative Commons licenses.

What matters is that the creators of DuckDuckGo promise not to collect or share users’ personal information. They also promise to block third-party trackers and encrypt traffic. Despite all the attention, nobody has proven them breaking the promises yet. There’s also a GitHub repository with the project’s open-source code for extra transparency.

Alternatively, there a search engine called YaCy that went all-in with the ideas of open source and decentralization. The service isn’t run on some company-operated servers but by the users’ own machines. Users allocate part of their computers’ memory, bandwidth, and processing power to serve the needs of the network. In return, they get a truly independent search engine no government can ever shut down or censor. It is somewhat similar to Tor and Bitcoin in that regard.

YaCy search results page

The catch is that YaCy isn’t exactly user-friendly. Those willing to enjoy it will have to deal with network ports and a whole bunch of settings, but there are well-illustrated guides to everything users, webmasters, or developers might need. YaCy client versions are available for Windows, Debian, and macOS machines, but you can always recompile the code for whichever platform you prefer. The open-source code for the project is available on GitHub.

There is also a selection of meta-search engines like an open-source project called SearX. These fetch results from other engines like Google without disclosing users’ data. In the case of SearX, users run their own instances of the service with custom settings like the list of search engines to pull the data from.

SearX search results page

It feels like any other search engine and has all the basic features, but before using it, you will have to choose a public instance to connect to or create your own. Since public instances can be run by anyone, you should be careful as the instance administrators may be able to log and share users’ data.

This isn’t a particularly exhaustive or detailed list, but the point is that there is a whole market of privacy-focused products. Some are clunkier than others, some are less known, but there’s a lot to choose from. If there’s too much, just go with DuckDuckGo and don’t skip the last section.

Privacy Is Kind of Hard to Keep

Keeping your privacy is a complex game. Getting the search engine problem sorted is just one part of it and it is virtually useless without a wider strategy.

Whatever you search through a private solution like DuckDuckGo will still appear on your browser history. Your ISP will still know at least the domain names of the sites you visit, if not the actual pages with timings and location data. There are hackers and vulnerabilities.

But there are a few measures to take before moving to some isolated forest to live inside a tree.

There are privacy-friendly browsers like Brave that will keep an eye on your extensions and website scripts. There are numerous VPN solutions to choose from, which will reroute your traffic through some other place in the world. There are private messengers, email services, and all sorts of ad and tracker blockers.

On top of that, there are common-sense practices that keep you on the safe side: having different strong passwords for different services, using two-factor authentication, updating your devices often, being careful with what you share on social media. These add up.

Prioritizing some aspects of privacy while neglecting the others will not help. Unfortunately, it takes time and effort to set up all these solutions and remember long weird passwords. Things like VPNs and password managers cost money. Then you have to keep on doing all that for as long as you want to keep private. And then there’s the trap of achieving peace of mind after simply installing a private search plugin into your Chrome.

But, arguably, the ordeal is worth it.

This article is a part of our Occupy the Internet series, where we review the current trends in the nascent decentralized web and cover the burning issues of privacy and censorship.

Follow us on Twitter and Facebook and join our Telegram channel to know what’s up with crypto and why it’s important.

Found a typo? Highlight text and press CTRL+ENTER

Subscribe to our Newsletter

Tags: decentralization, Google, privacy