Ask HN: How to do literal web searches after Google destroyed the “ ” feature?
I used this quite frequently but since Google """"improved"""" it last year (there was a popular HN post complaining about this) it doesn't work anymore. Search for a domain name with quotation marks for example just recombines the contents of the domain and returns a bunch of unrelated content completely cluttering what I am looking for. Until last year it used to return no search results if there weren't any exact matches, which is the whole point.
Does someone have a work around for this phenomenal Google decision?
Comments
Just give up on trying to make google work. Google has definitely not improved. Recently I tried searching "hugginface madebyollin" (with a slight type to hugginface instead of huggingface) and it literally didn't show the obvious result. https://www.google.com/search?q=hugginface+madebyollin. I've switched to duckduckgo and couldn't be happier.
> Google """"improved"""" it last year
Google's done a lot of "improvements". I hate to say it but its quote feature has been broken for a decade. You're only now noticing?
I've tried using other search engines. I've settled on DuckDuckGo. It also does not have a working literal-quote feature. But it's much less infested with SEO garbage.
I switched to DDG because Google broke literal quotes years ago (then fixed it, and broke it again?). Now its broken on DDG. Considering Kagi.
I stopped using google search because it is infuriating.
[flagged]
site:domain.com is how you do a site URL restriction, if that's what problem you're having.
Enable verbatim mode to make quotes work better. Even then I don't think it's absolute.
[flagged]
Do you have an example query? Quotes for literal words work for me. Domain search was always with with site:domain.com and not quotes and works fine. I think bing/ddg removed that feature recently?
What's an example query?
Quotes don't work but
>>>>Search for a domain name with quotation marks for example just recombines the contents of the domain and returns a bunch of unrelated content
I believe if you search for site:domaingoeshere.com yourqueryhere
That will spit out results only from that domain. I think that still works ?
Weird, I still use the quotes to search for phrases, IDs and other things and it works well. Never noticed they changed anything.
Any specific examples of queries you wish worked differently?
Brave search is pretty good. I don't like their browser but use their search with firefox.
Huh? It's still working.
really depends on the query, I have seen it working and I have seen it being totally ignored.
When it is ignored I see a link below the search box to search for the "exact phrase" instead.
It seems like more of a suggestion recently. Often, things I quote aren't even in the first page of results.
> Search for a domain name with quotation marks for example just recombines the contents of the domain and returns a bunch of unrelated content completely cluttering what I am looking for.
site:example.com and -site:example.com still work, I think.
"example.com" also seems to work perfectly well.
That limits results to pages on example.com (or neg for the second) but that's not what OP is trying to do. Searches for "example.com" should return pages where the exact domain "example.com" appears in the page. It does sort of work for "example.com" itself, because it's so common presumably, but doing it with other domains that are less common is far less likely to produce anything useful.
I use a local hosted web spider and search engine
It only index the pages that I've viewed so it's not spammed by SEO junks
Can you give an example? Searching for
"news.ycombinator.com"
How do the search results for a site: query compare to just quoting what appears to be a DNS domain containing punctuation tokens?
inurl:news.ycombinator.com
site:news.ycombinator.com
"news.ycombinator.com"
news.ycombinator.comKnowing Google, it's gaslighting you^W^W guessing that you actually meant to type the quotes for that one search.
Bing. Not that it works well. Also, spending less time on the computer. The more time you spend on the computer the more money google makes. They don’t want to help you find the thing you need, they want to do the opposite so you spend all afternoon searching so they can sell more ads.
Similar vein, how the hell do I search an image and combine it with text anymore? I hate Lens and there's apparently no way to properly "image" search anymore.
Yandex's image search is pretty good
Have you tried using "Tools -> Verbatim"?
+1, this is the only way to get old-school(ish) results, even without explicit quoting. Unfortunately, it can't be enforced just by adding parameters to the URL and has to be selected manually every time. (edit: actually it can, see below)
Verbatim really highlights how almost useless the default mode has become.
You can add it with parameters: tbs=li:1
https://www.google.com/search?tbs=li:1&num=30&safe=off&q=testAwesome, I stand corrected.
But that's almost worse than removing it completely! They recognize that people still need to do this, but deliberately make it more difficult in the UI.
(I worked on the team that added the tbs= parameter, 14 years ago.)
There's nothing nefarious here. The integration of &tbs= is deeply tied into GWS; it's written into the C++ substrate, while most of the rest of the server has been rewritten in Java + a bunch of custom languages. Most likely they simply can't remove it - it's permanent technical debt that nobody has the expertise to touch anymore.
Changing the UI is all done in a plugin these days, and is a relatively trivial change.
> (I worked on the team that added the tbs= parameter, 14 years ago.)
Thank you
Like ChoGGi said, you can use tbs=li:1 You can test the other parameters here.
https://serpapi.com/playground?q=The+most+popular+fruit+in+t...
Other useful parameters, you can use nfpr to force it to not correct your search terms
https://serpapi.com/blog/filtering-google-search-and-google-...
It's never stopped working for me, for all queries, and believe me, I use it very often.
This post reeks of Kagi spam to me.
Sorry guys, your product failed, no one is going to pay for search no matter how much Google sucks. Move on.
I agree that as a programmer who frequently needs to search for long literal strings verbatim, Google has become notably less useful than it used to be.
I wonder if there is now a gap in the market for some kind of "literal" search engine that makes no attempt to infer meaning on your search terms and simply gives you the closest results? In other words Google ca 2012.
For technical stuff I've long replaced Google search with phind.com even just pasting so.e generic error spit from the console works.
Have you tried grep.app ?
https://publicwww.com/ is a great tool for this, though the size of its index leaves a lot to be desired. Still, for enumerating well-SEO'd homepages that use a certain tech stack, it's quite useful!
The minus (-) operator isn't a hard and fast rule anymore either. It's particularly galling when you want to search for something that's similar to something else that's very common.
For locking to a particular domain I had always added `site: example.com` to the query, rather than adding domain to a double quote statement.
I have used double quotes to limit to a particular _phrase_ as recently as last week. I'm not privy to the improvements you mention, do you have a link?
The site:domain.tld syntax should not have a space between the colon (:) and the (sub)domain.
Incorrect:
site: domain.tld
Correct:
site:domain.tld
The - modifier to exclude a word does not have a trailing space.
Incorrect:
- site:domain.tld - query
Correct:
-site:domain.tld -query
I’m less certain about excluding a verbatim phrase, but I believe you can do as follows.
-“exclude this query string”
While this space rule still applies, so don’t insert a space, although this needs more testing by me to know for sure that this exclusion works properly.
- “this is not correct syntax imo”
Kagi.com
yandex.com duckduckgo.com
Piracy has basically been purged from google. Duckduckgo isn't much better, since it's Bing on the backend.
Instead, use Yandex. It returns direct relevant hits and just bloody works. And if you're in the USA, you're also shielded by the fact that Yandex wont share your searched etc with the USG. You know, being a Russian search engine :)
When this happened to me, I found better results on https://duckduckgo.com/
I’ve used DDG exclusively for a couple years now and at least with my usecases have found it better than Google in every single way other than images (which I still use it for, but need to do !g to proxy Google maybe 1 in 5 times).
FYI: !gi on DDG for direct Google Images search.
I find better results on it in general.
I use Google as a fallback but these days it happens perhaps once every couple months, and mostly I don't get anything out of it.
Yes. IMO DDG has been better for general questions for a long time.
Google remained better for programming questions for significantly longer (I speculate this may be because Google's own programmers used it, and complained when the results sucked :-)), but now it's not. Not really.
Like you, I still use Google as a long shot, but that's become quite rare.
I sometimes use Bing for Microsoft-specific questions, if DDG doesn't give me what I want. I have the sense that Bing covers Microsoft a little better than the others do. I have no real solid evidence for this, but it seems plausible on the surface.
It depends on the usecase. If you're querying for something local like a restaurant or a store, google wins by orders of magnitude. If you're researching, duckduckgo is fine. Duckduckgo doesn't really compare to Kagi though.
What's wrong with DDG and local searches? I find it exceedingly good, especially with the country selector being so up and front. It's very easy to search for those kinds of things in other places, for example to plan a trip.
Kagi for me is a no-no. I mostly browse on temporary containers, and also have Firefox delete cookies on close. A search engine that requires me to be signed in would be a massive pain to use.
>What's wrong with DDG and local searches
Maybe its different in the US or just not optimised in Germany, but the quality of search results isn't comparable.
Query: "All you can eat Kiel"
Google:
1. a map with the top three restaurants for this context in the foreground
2: a list of restaurants, sortable by price/date/hours
3 to end of page: links to homepages of the top restaurants
DDG:
1. a completely random assortment of irrelevant links on the left
2. a semantic web box with a 3/5 rated irrelevant restaurant on the right
3. wild youtube-videos vaguely related to the query between text results
4. images
5. more wild links
that kind of thing. And Kiel isn't a small city, we even had DDG print ads here.
With all respect, there’s not a lot of restaurants around here ) comparing to i.e. Hamburg. A couple of steak houses, some locals, 4-5 asian food and the Brunswik, which is always busy. There’s also Kiellinie, but it’s mostly a seasonal thing. Though there’s a lot of folks around sipping cocktails with ice in winter, crazy. Love this city.
I'm not in the US. :)
When I try your query I only get a bunch of local German stuff (including a map with restaurants).
I admit, I don't even know what Kiel is so I can't validate the results but despite them being in Germany (and my search having had the country selector set to UK), it looks pretty good.
If I search something more generic like "all you can eat sushi" I get a neat map with a few restaurants that do that, plus a bunch of TripAdvisor and other directory and blog posts exactly about that, all local.
If I try your query again, setting the country to Germany, I get https://yummy-kiel.de/ as the first result, a map with top-three restaurants as the second, and again, a bunch of directories and blogs with all you can eat Kiel.
Seems pretty good to me.
Duck Duck Go has similar problems for me. It's also recently started sometimes ignoring the "-" when you try to exclude a word.
We put out a partial fix for that recently and a more complete fix is forthcoming.
It's crazy how often you're in the comments of HN articles just waiting for a DDG shoutout haha it's appreciated!
Thank you! As a heavy DDG user, PLEASE don't become Google. Stay great and private.
Partial fix for the minus or the double quotes? Both are such a critical part for searching anything these days, it’d be a real shame to lose either.
Both.
That'll be awesome. The parts of the web that don't respect operands feel awful.
Having a minus act like a plus has been particularly tough (for years now, I believe).
I think the "-" is case-sensitive sometimes. So if you write "-honey" it'll still return "Honey" results, so you have to write "-honey -Honey".
No, it just doesn't work at all. Search /r/duckduckgo for "- operator" to see tons of threads.
DDG has started injecting completely untreated recent news stories to the results of a search with few hits. Like searching for a programming error with specific class names will give you several articles about the woes of the Democratic Party in recent days!!
Same for me, but with location-related stuff. I often see random Wikipedia articles or tourism websites for stuff near the city my IP address is from... Worst of all, sometimes there's a block of filler results between actual results, so I sometimes give up and don't see some of the results that might've been what I was looking for.
Same here, yesterday I searched for a very specific error message (don't remember exactly), and even on the first page there were local news stories. This was on Metager, which also uses Bing as one of its backends.
I think it's Tay v2.0 injecting itself into the conversation.
Are you sure this isn't from SEO on the part of the news sites? I have not seen this at all.
Thank you for asking the important questions, you've saved me a large amount of time here realizing that this isn't just my particular case of search terms being broken somehow. I guess we've all counted on google being reliable all these years and it's kind of a shock that they've gone and tanked their own usefulness after building up like twenty years of trust. Rude.
You have to use a special "verbatim" search product from Google, it isn't the main search box anymore. Look under Advanced or something.
I thought this was a joke.
What it is like anti-user design. Why take away a decades long precedent to hide it in a menu?
Great tip! Seems like this adds the query param "&tbs=li%3A1", so this might be something you can configure as an extra search engine in firefox. But then I am a happy kagi customer, and I was just thinking that I can't recall the last time I had to do a !g to find something. For me, google search si pretty much dead. I only use it now when on someone else's pc
Sounds like a good case for an extension even! "Auto add this when I go to google"
You can also adjust your browser's search engine defaults.
For now, this seems to work: https://www.google.com/advanced_search but, given the pace at which Google "improves" things, I don't have much hope that it will last.
That page has been there forever. Here's the version from Dec 2000: https://web.archive.org/web/20001202041000/https://www.googl...
First of all, wow. It's there (under Tools, you don't even need to go to Advanced), it works, and yet I've never once seen anybody talk about that feature's existence on HN until now. And people have been complaining about not being able to do verbatim searches for years and years here on HN.
Second, I just looked up when this feature was introduced (assuming it was fairly new), and it was in... November 2011. It's been there for the past twelve years. See:
https://www.wired.com/2011/11/google-verbatim-search-back/
https://searchengineland.com/responding-to-complaints-google...
https://www.searchenginewatch.com/2011/11/18/google-introduc...
Thanks for letting us know! It's been right under our noses the whole time -- and it's not like the Tools menu is even particularly hidden, at least on desktop.
> I've never once seen anybody talk about that feature's existence on HN until now
Every time I tried it, it didn't work.
It certainly changed a lot on those years, but the reason nobody acknowledges it is probably because it's a coin-toss if Google wants "verbatim" to mean verbatim today.
Quotes actually stopped working (they became a hint, instead of filtering the results) a long time ago, and many people insisted for years that the verbatim search worked. Probably because those tried it on the days when Google decided to use a standard dictionary. Nowadays even those people gave-up.
Yeah, I've never had much luck with verbatim search either.
Can you (or anyone) supply an example where it doesn't?
I just tried a whole bunch of queries on seemingly generic-sounding sentences from old pages on niche blogs and it found every single one.
So I'd love to understand when it doesn't work and if there's some pattern -- like if certain punctuation trips it up, or if it's simply pages not indexed in the first place.
It didn't work for any query. A verbatim search was very similar to putting every word in quotes. That's very obviously not your experience, so it's either a personalization thing, or Google silently changed it. Anyway, there isn't a pattern to find, it works or it doesn't.
But then, I've seen it work too. And stop working again.
OK, so can you supply any query then? Any example will do.
This is just about verifying the behavior, that it's not finding a verbatim result that it should. And it has nothing to do with personalization -- it's easy to run these logged out.
Also, "A verbatim search was very similar to putting every word in quotes" is not an example of it not working. The question isn't whether they're similar -- your claim was that it "didn't work", which I take to mean it was returning incorrect results, that correct results were missing.
I think OP's point is that this isn't a deterministic system for a given query. The feature flags set for the particular user (are you in the verbatim or no-verbatim branch of today's AB test), and the timestamp are also inputs.
There's been lots of talk about it over the years: https://www.google.com/search?q=google+%22verbatim%22+site%3...
I think I also mentioned it a couple of times when the same complaint came up.
The problem I think is that "verbatim" is not a word that one thinks of, so nobody searches for that. Plus it's hidden in a generic "Tools" menu. Sometime you get a link to search for the exact phrase at the bottom of results, but that too is subtle.
On the search results page, there's a "Tools" button in the upper-right that expands two dropdowns. Change "All results" to "Verbatim".
Wow, that's a good tip.
Here's a URL Link to the advanced search: https://www.google.com/advanced_search?hl=en&fg=1
I tried it and it looks like it does nothing but wrap my query in quotes… and then return non-exact results if not found. How is this any different than what people have historically done?
That isn't different, Verbatim search isn't on the advanced search page. You can only get to it from the search results page like in my earlier comment.
wow, thanks! how did you find this link? I don't see it anywhere in the main interface.
It's not on that page, you can only switch to Verbatim on the search results. OP was mistaken.
> You have to use a special "verbatim" search product from Google, it isn't the main search box anymore. Look under Advanced or something.
I actually use that, but it has its faults. You get more spam results and iffy sites (e.g. Wikipedia clones). It's also missing some of Google's convenient features (like doing unit conversions and arithmetic).
IIRC, Verbatim mode is closer to the raw results of Google's underlying search engine, before some of the massaging they do. Some of that massaging is bad, but some of it's all right.
Why would they make their very expected functionality something you have to dig around to find now? That just seems like really bad decision making that should have been spotted by someone on top and screams that the people on top are now disconnected from reality.
I swear that it used to work for certain strings I'm trying to find now which I was able to find information on and now it isn't even returning, with "verbatim" set, something that is in a very well-known program's documentation. Bing finds like three results. Google has dropped the ball so hard it's embarrassing
My guess (complete speculation, but I worked with the Search Quality folks about a decade ago):
1 - By interpreting your search, it leads to better "search quality" by having one model say "i think this is what they want" and another execute the search. P90 accuracy is increased at the cost of P99 accuracy.
2 - If you search for a literal string you know exists, you expect to find it. By interpreting, fewer search inputs with literal strings make it to the search function.
3 - Since Google is interpreting more searches, this gives ad-placement a route in to favorably interpret "they want to buy something" even when this isn't the case. This makes Google money.
4 - People that used to use literal searches either stop, learn how, or switch search engines.
5 - After a couple years, business metrics show that literal searches represent 0.1% of queries and make less money, is it really worth investing in? When it was a P99 issue it fell off the radar and now the P999 is lost.
So a series or rational decisions by rational actors leads to a decline of a used feature because of business incentives and chasing P90s at the cost of P99s.
So they're basically just as good as some malware search engine that inserted itself into grandma's computer now
Fantastic
At least those entertained people.
I couldn't get my fiancé's mother to stop re-installing bonzi buddy or whatever that toolbar was called because she 'loves the purple monkey, he's so cute!'
Man that's crazy in the early 2000s I was volunteering with developmentally disabled adults and had to explain to them how bonzi was a 'bad monkey' and that he was 'not a friend' and they got it
It's also possible they they're correct, and the average user of today has much less need for verbatim searches than the average user when the search engine was first designed.
This is definitely the case. Sit down next to a casual user one day, and you'll find that 'verbatim' is the absolute last thing they need.
Nah their product is broken and I'm going to avoid using it from now on except when I need to find something on a map or mess around on SEO for work
If they break a key feature to MAKE a verbatim search happen with literally two keys pressed, and are apparently not even indexing what they used to anymore, they're dropping the ball. Most people know the quotation trick now and are probably assuming it still works
The quality of search results I get from Google has seriously degraded over the past several years (I think about the time they decided to start interpreting my searches rather than just search for what I asked for). From my point of view, verbatim searches are more necessary now than ever before, to work around that issue.
I tried using verbatim searches to make Google decent for me again, but Google defines "verbatim" rather differently than I do.
This is almost certainly the case. Those of us who learned to use and mastered keyword-style search are the minority. Most people search whole phrases, expecting a contextual answer.
Yeah, it's Google. I feel like, at this point, Google is basically synonymous with "bad decision"
Use Bing? I've found for code/error message snippets in particular Bing is often better, while Google will incorrectly guess I mean some related but different thing and serve the wrong results.
Doesn't seem to work. Search for "internal engine" and see most results talk about internal combustion engine. By contrast, Google with Verbatim option returns only results with the exact phrase "internal engine".
Bing's verbatim feature needs a + in front of the quotes, I believe; but then it will still screw up horribly on things like error codes (you want 1234, it'll give you 1235 and 1233.)
confirmed (although I use ecosia instead of bing directly). Now its like:
semantic search + realtime needs = google;
raw searches is ecosia(bing)
and most needs are covered via GPT4 actually. Error debugging + code snippets, general interest explanations, arguing things, evaluating things interactively. Basically everything I don't need hard facts of the last 2 years for.
> I used this quite frequently but since Google """"improved"""" it last year (there was a popular HN post complaining about this) it doesn't work anymore
That's not my experience? The quotes still seem to work for me? Do you have a specific example? And / or can you point to said HN post?
works for me too, maybe being in europe does the trick?
I just tried https://www.google.com/search?q=%22I+used+this+quite+frequen... and it works for me now, but I definitely recall having problems searching verbatim before, perhaps they improved/changed it recently?
I can confirm the experience that sometimes searches for quoted terms yield pages that do not contain the quoted term exactly. I don't have a specific example from the top of my head.
Is it possible that the results simply don't exist and instead of giving you nothing it gives the next possible match?
It could be this - but you would notice it after opening the search result.
I spend some time reading a random website until I realize the sentences i was looking for is not there. Useless and frustrating.
[dead]
Kagi. Never have I been so happy to send someone $10 every month. When you become the customer, not the product, it’s amazing what can happen.
I tried it a little but honestly thinking about having a limit of searches made me anxious every time I thought about searching like “is this really necessary” and so I went back to google.
Kagi searches are soon to be unlimited (fair use policy?) for the $10 plan. If I understand correctly, this may happen around October 2 when other new features are released.
I use search a lot in my workflow and my usage is showing 2k/month. I expected it would have been 3-4k.
Why not just ignore that the limit exists? If you hit it, you can always fall back to another search engine, after all, so it shouldn't be cause of anxiety.
I can offer that I am constantly online and using Kagi across my devices for all searches and so far have not come close to using up my quota.
How does it fare for bilingual users?
Does anyone knows how Kagi performs on country specific search or even search in another languages?
It works fine in French for me. I really like that I can have international search by default, and specify a country when needed using a bang like !fr.
Working fine for me with Danish as my primary language. But I think that I mostly search for English content, so unsure how your experience will be.
But could you not just pay for a month and try it? I don’t think their plans bind you for a longer than a month.
One personal anecdote. It did not do that great for Norway/Norwegian, I ended up switching back to google when trying to find some stuff in stores.
For recipes and stuff it was fine.
“ Our data includes anonymized API calls to traditional search indexes like Google, Mojeek and Yandex”. They pay google to do this?
I believe DuckDuckGo does (or at least they did) this with Bing. Starting a new scraper at a scale that users would need to be useful for what they're used to is such a huge jump. I'm sure if Kagi continue to grow they'd prioritize their own scraping too, but that's just not feasible at first.
Back in the day I'd suggest doing it via Alexa top sites, but now that Alexa is gone, I'm not sure what strategy I would use, but I would want to hit sites that are like the "top 10000 most popular" first, and scrape every inch I could.
I think Kagi is going in the opposite direction: https://blog.kagi.com/small-web
They try to highlight small, personal websites instead of the big mainstream sites.
(This was a HN submission 2 weeks ago)
I saw that, but that's kind of useless when you kind of want something like SO or similar results, something Google keeps failing at.
that's what that means yeah, but not necessarily present it in the same way
100% agree on Kagi. Happy customer. Thought it would be just another one of my attempts to use Duck Duck Go that dies after two weeks of !g usage. Turned out Kagi just works. The biggest improvement / gains is on mobile, where you suddenly don't need to scroll through 5 screens of ad results to get to the content.
Yeah, me personally I am more upset with how irrelevant and bad Google search results have become than I am worried about privacy. I know and have accepted that Google invades my privacy, but the trade off used to be I could find whatever I am searching for, but now I can't find anything on Google and it has made my job so much harder.
The Google search algorithm from 5 years ago was amazing, why they decided to change it for the worse is something I will never understand! And no I do not blame SEO entirely since that existed 5 years ago, what I am often looking for but can no longer find is information that has nothing to do with any products. It's not ads that I need to page through, but unrelated and bad results that are limited. I do not want to see the same results from page 1 on page 3.
I wonder if part of why its better is due to other users providing feedback about results, but also you can pin results from specific domains to the top. Like I can pin any results from StackOverflow, instead of the garbage StackOverflow rip off sites Google keeps giving me, its pretty obvious its ripping off SO because I just read the same thing word for word on StackOverflow three links ago. Thanks Google.
> I wonder if part of why its better is due to other users providing feedback about results, but also you can pin results from specific domains to the top.
I think that matters very little. Kagi had excellent quality results for me from the start, in a huge variety of topics in several languages. Their user base is probably 90% American hacker, and I'm getting good results on queries they would never use.
This.
Kagi is incredible and worth every penny simply for being able to remove the SEO scam and tire fire that is Pinterest from all image search results.
You sold me.
Fuck google, I have work to do. Thanks for the tip! Nice realizing that they've basically been wasting my time for a while now and that there's a decent alternative available.
Kagi copied the same lame-ass google search experience.
Search engines should be able to support even those who are not SEO experts and not the first ones to arrive and sit on specific keywords. What I mean under that, if you have multiple good, exhaustive answers for a query, why not offer varying/random good results, so every link would have a chance? Let people break out of their bubbles.
Same power pyramid scheme. Yukk.
Can’t second this emotion hard enough, love it, have never looked back, almost never bail out to !g - still use g maps for most location stuff, but all my web search is very comfortably living on kagi
100% ^^this^^
I would love to see some of these privacy-focused providers like Kagi and Tutanota/Protonmail align themselves into a "bundle"-type offering (think video game humble bundles) where for like $35/month you got access to a bunch of useful tools like this. It would really expose a lot of people to services they aren't super familiar with already, so even if it was at a slight discount to the provider, they would acquire a lot of new users I bet.
Indeed. I'm tired of data brokers masquerading as other things (search, email, whatever).
Signed up for Kagi today and have been looking for a permanent gmail solution.
Had this bundle been available today I'd have jumped on it.
This is the answer. I’ll never use Google search again.
Being logged in while making search queries in search engines poses significant privacy risks. The searches can paint a comprehensive profile of the user, and these data often remain stored for extended periods. There's a chance this information might be shared with third parties. Coupled with other user data, these logged-in searches can pave the way for targeted advertising, sophisticated predictive analysis, and potential exploitation by governments or malicious entities. In the event of data breaches, the user's logged-in search histories can be exposed. Furthermore, users typically don't have clear insight into how their data is utilized when logged in.
I hope Kagi introduces an anonymous access feature. For instance, it could incorporate zero-knowledge proofs (ZKPs). These are cryptographic techniques where one party (the prover) can confirm to another (the verifier) that a claim is accurate without disclosing any additional information. This is especially beneficial for authentication scenarios where it's essential to avoid sharing extra details.
To implement zero-knowledge authentication for quota API access:
1. Token Creation:
- Each month, users receive a token tied to their identity and quota.
- The token can be split for use on multiple devices using cryptographic methods.
2. API Access:
- Clients present a zero-knowledge proof (ZKP) to confirm they have a valid token and haven't used up their quota. The server verifies this without seeing the exact details.
3. Client Synchronization:
- Each client tracks its quota usage.
- Synchronization can be peer-to-peer or through a centralized, encrypted server to prevent double spending of the quota.
4. Quota Renewal:
- Monthly, old tokens expire, and new tokens are issued.
Challenges:
- ZKPs can be resource-intensive.
- Token security is crucial; there should be a way to handle lost or compromised tokens.
- The system should prevent quota "double-spending" across devices.
- If a centralized server is used for synchronization, it should operate with encrypted data.
This way Kagi would only know who their customers are but not what kind of searches they make.
I'm not searching for anything terrifyingly illegal, and for the rest Google and MS already scrape and compile every byte of data I've ever generated. Why would it suddenly be a problem when a more reliable and less vicious company is doing a fraction of that?
You have to understand that most of us aren't fighting some battle for "perfect privacy," I just want a search engine that works for me, rather than advertisers, at the level of the search results themselves.
What is counted as "terrifyingly illegal" changes without a moments notice on the whims of your rulers. So even if you're not googling on how to bomb the government, there are hundreds of other subjects and opinions that could in the future make the majority of your neighbours, family and workmates think you deserve to be shunned, fired, in prison, or worse. That is why people want to protect their privacy.
Ok, but again Google and a hundred data brokers already scrape every detail of my life no matter what I do. Even if I become a hermit in the woods the existence of my friends and family who don't take those precautions would make my efforts worthless. Meanwhile we're talking about Google/Bing vs. Kagi... not "Super Secret Perfect Privacy Magic" vs Kagi.
So while I understand your overall concern, that ship has sailed for search engines and the internet. We're living in a world full of networked cameras that people voluntarily and happily install, of people broadcasting their lives 24/7. The idea of perfect privacy is getting downright mystical/religious.
Sure, and for what it's worth I trust Kagi. But I can understand those who are more strict with their privacy.
In the end I think we have to accept in some way that everything we say, write and do is subject to surveillance, and that the government might want to kill you for any reason that you'd have no chance of preventing.
I get your perspective. A lot of us just want a search engine that serves the user first, not advertisers, especially at the results level. It's about function over strict privacy for many--everyone has their own privacy threshold.
But it's also about digital data autonomy. It's not just about avoiding surveillance over sensitive searches, but having control over our data's destiny. Even mundane data, in aggregate, can sometimes be used in ways we can't predict.
Personally privacy is a strong concern for me; I have many aspects of my digital life set up less conveniently in exchange for privacy.
In this case though we've have on one hand a product that definitely does aggregate data about searches, and doesn't do what I need very well; and the other a product that could, but does not currently aggregate data, and does an excellent job serving my needs.
And importantly there is no option of a product, available now, that is verifiably prevented from aggregation. Even a VPN unless I disconnect and get a new random IP between every individual search does not provide that protection. (And then browser fingerprints even.)
I’ve gotten tired of these boogey man arguments.
There are sooooo many other ways to fingerprint than an account.
Oh look, this MacBook with X by Y resolution from this IP address has had 100 searches for the past 2 hours. Oh no! He switched to incognito.
Kagi already provides a way to search anonymously via a random email address (we do not really verify it or need it for anything) and Bitcoin/Lightning payment [1].
Since you are interested in cryptography, there is a discussion on Kagi feedback site along the same lines as your idea, about possible ways to achieve this without the need for cryptocurrency. [2]
[1] https://blog.kagi.com/accepting-paypal-bitcoin
[2] https://kagifeedback.org/d/653-completely-anonymous-searches...
Thanks for the links. Using a disposable email with crypto payments and occasionally generating a new account to unlink from previous searches could be a viable intermediate solution.
Also, I found this link [1] in the thread you mentioned. They seem to have implemented something like that.
Just to make it clear, Kagi does not link searches to an account already, to begin with. Refer to our privacy policy [1]. We simply do not need that data for anything and it would be just a liability for us. Our philosophy is that users should personalize the search feed themselves and this is why we built features like the aiblity to block or promote domains, create search lenses and many more.
However there is no technical way of proving it. So cryptocurrency and cryptography are ways to achieve anonimity from a perspective of a user, regardless of what we are doing.
Any system that can check balance, can link searches to a user. There's no way around it. In your case, Kagi would need to trust the client with the balance, which would be insecure.
There's only one solution, and that is that you need to put a bit of trust in Kagi. Compared to the major one, Google, you can chose between one that promises to not store data, and one that promises it does (and does a lot).
It's always a bit sad that here on HN, when companies try to do better than bigger players, there's always people who think it isn't enough. It has to be absolutely impossibly perfect.
> Any system that can check balance, can link searches to a user.
For what it's worth, you can buy a physical Mullvad gift card and use that to create a very anonymous account for VPN use.
Even if you buy your gift card from a major online retailer, it comes from a stack of gift cards, nothing tracks which one was sent to whom. You can also exchange gifts among friends.
I'm not a cryptography expert, but from my research, shouldn't it be possible to verify quota on ZKPs server-side? Essentially, the server doesn't need to know the specifics of the user's identity, just that they possess a valid token and haven't exceeded their quota.
You can use search engines like Google without being logged in. When combined with tools like uBlock Origin and Cookie AutoDelete, it becomes more challenging for them to build a singular profile about a user, especially one tied to payment methods such as credit cards.
I genuinely appreciate what Kagi is doing, and I'd absolutely be willing to pay for their service, because if you're not paying for a service, you're the product. I trust companies to uphold their privacy promises, but "Trust is good, but proof is better." ;)
The issue is implementing it client side. ZKP means that you cannot simply embed a token in the URL, but instead need to participate in an active protocol. You could implement this in JavaScript, but then you need to trust the JS being served from the server.
Even once you do that, you have all the other tracking mechanisms that the server could use if it wanted to.
They key word is server side. You have no way to verify that they are not tracking sessions as an user.
> Any system that can check balance, can link searches to a user.
I don't think it's true. I can immediately see at least two ways how it can be done without identifying the user.
1. Each user gets X tokens at the beginning of the month. When searching, user supplies a token, which is immediately burned. The token does not contain the user identity, just signature validating it's a valid token.
2. Variation of the above: each user gets a token good for X searches at the beginning of the month. When searching, the system will return a token good for N-1 search each time token good for N searches is presented. Again, no need to contain user identity anywhere in the system.
Of course, both solutions have their downsides (sync between multiple devices, stealing tokens, losing tokens, etc.) but it id definitely possible. And I am sure if somebody spent a little time thinking on it, these ideas can be seriously improved to eliminate the downsides without introducing the need to identify the user.
In both these cases the search engine provider could easily store your identity together with your token while issuing it and recover the identity once the token is used without any way to prove this from the outside. They could even issue tokens in the form AES_ENC("SOME KEY ONLY THEY HAVE", USER_ID | counter) and you would not notice. You would have to trust them that they won't do this, which is no improvement to the current thing Kagi does (saying they won't collect any data, while admitting they can't prove it, you just have to trust them).
I think there's a fundamental difference between "X can not be implemented" and "can we trust this provider to implement X correctly"? In this case, it can be implemented without violating privacy. But of course you need to trust them to actually implement what they say and not instead put 9000 trackers in each page and track your every movement like certain other big companies do. But these are different things - the comment upstream claimed that the subscription system can not be implemented with privacy. This is not true - it can be. Whether or not a particular provider would implement it, and whether we can trust them that they did - that's a different question, which is also important but does not change the answer to the original one.
> Being logged in while making search queries in search engines poses significant privacy risks. The searches can paint a comprehensive profile of the user, and these data often remain stored for extended periods. There's a chance this information might be shared with third parties. Coupled with other user data, these logged-in searches can pave the way for targeted advertising, sophisticated predictive analysis, and potential exploitation by governments or malicious entities. In the event of data breaches, the user's logged-in search histories can be exposed. Furthermore, users typically don't have clear insight into how their data is utilized when logged in.
This reads and smells like ChatGPT / AI.
Was thinking the same thing. Not even gpt4
Another happy customer here as well. I use at work and home and plan to start using on mobile browsers also.
The ability to essentially "weight" particular domains (pin, block, or anywhere in between) has saved me so much time. There are certain searches I do (music-related in particular) where I always want particular sites (metal-archives, bandcamp, etc) to be the first results, and having that as an option is great. It means that searches that I perform often have a result within the first 1-5 results that is exactly what I want.
No ads, way less SEO spam, and the ability to completely remove domains from results if I think I need to tweak it further. For most of my searches I previously used Google for- Kagi makes Google's results look laughably bad.
I've also been using it at work for tech searches (linux, redhat, etc) and it has saved me time there too.
I use a "family" account- and have one work account and one home account that way I can have different settings for different environments (would be neat if this could be built into non-family accounts though... like "personas" or "profiles" or something...) because I'm overpaying a bit to have the two account setup and don't reach the search cap. I think I'm okay with that though, because having the cap so high means I've removed the "running out of searches" anxiety from my usage of the service.
I’ve replaced most of my google searches with LLM discussions and some bullshit checking
I usually only need to understand a concept, not understand if the personnel and company names it made up actually exist
everything else I use google for are just addresses
so I’m wondering if a paid search engine would shift my behavior back to search engines, or if that ship has just sailed
LLMs can not yet replace good web search. There are whole categories of queries whera a LLM is more or less helpless with. Think navigational queries, shopping /reviews, location aware, 'grep the web' style queries just to name the few.
For example:
nyt crossword
cheap iem reddit
starbucks near me
M7FFALP
Likely, a good search product in the future will be a combnation of both.
have you tried this on phind.com? it'll create multiple queries to find and organize data and it's very good and saying it doesn't know something rather than give a BS answer.
> everything else I use google for are just addresses
> LLMs can not yet replace good web search ... think navigational queries, shopping /reviews, location aware
amusing.
> I usually only need to understand a concept
You should know by now that LLMs will and do lie in subtle ways that are not apparent to non-experts. Using them to understand complicated concepts is a great way to "learn" incorrect information. To be fair, the same can be said for humans, but humans are worse at bullshitting.
I’m aware
I really just need to converse about a topic for more inquisitive-ness and to form structured thoughts
It will tell me if I’m conflating concepts, before bullshitting about the ways theyre different. Thats fine, my blind spot would have been that I was conflating a concept for the next decade.
In that regard its the same or better than a human
I don't need it to be the source of truth, I need it to be conversational. It can make urban legends just like a person does, I don’t care, just give me a way to talk about a concept and decide if I want to learn more and it does that extremely well
Weird question that I have that I'd love anyone who makes a Kagi account to trial after reading the parent comment to answer:
When you make your account, you're given the option to customize. When you do, you can pick things like color theme and how URLs are displayed. On the right hand side of the page there is a preview of what your Kagi searches will look like.
In my example, the demo Kagi search is Magic The Gathering. I play a lot of Magic The Gathering. I spend most of my time online searching for things related to MtG or brewing decks, second only to things related to software development.
I imagine it's coincidence. MtG is a pretty nerdy hobby and Kagi seems like a pretty nerdy product. However, it made me uncomfortable enough to ask:
Is that what it shows for everyone? Or is there some tracking going on already that is being demoed? It's almost certainly the former given the positioning of Kagi in the search market, but I'd like to be sure.
Vlad here, Kagi founder. Also an avid MTG player. I came up with the idea for that preview.
Hi Vlad! It's a fun nugget. I just suspect I'm in the very slim subset of people who was a little taken aback by it.
We all need a dose of healthy skepticism.
I know it means nothing in the grand scheme of things, but y'all just got another customer.
I'd been on the fence but after reading through these threads and seeing a real reply from a founder (its been a while since I've seen an honest, non-PR Speak answer) I am excited to try y'all out.
This area was ripe for disruption with how terrible searches have become, especially on mobile. I'd be happy to see you eat G's lunch here.
I'm kind of blown away by how popular that game has gotten over the past few years in North America. I think the pandemic really accelerated the popularity of that and D&D, people are still doing these things after all of that. Even saw someone playing over the phone the other day. I don't seem to remember it being so popular but now it's more than ever and hardly a surprise tbh
The two biggest reasons for this are that the format Commander has specifically blown up in popularity and Wizards of the Coast making a first-party desktop/mobile client MtG Arena to compete with games like Hearthstone.
Commander is a 4-player casual format that's has as much in common with more typical board games as it does with traditional MtG.
Arena is likely what you saw someone playing on their phone. Funnily enough, it doesn't support Commander!
I think they were playing d&d over the phone but yeah it does look like it's alot more engaging than what I seem to remember in middle school
Yes it's what showed for me as well and I've never played or searched it. Just a coincidence.
Just as an extra data point - same here, got shown MtG results, never played it (though I might have searched about it a few times way back).
Do you ever go past the 1,000 search "limit"?
I'm considering signing up; it would be one fewer service I'm relying on google for.
I was quite afraid of this, but apparently I'm around 700-800 searches a month and I search quite a lot during my work.
Kagi is also working on removing the 1000 limit on the 10$ subscription and offering unlimited searches.
Switching from the 5 $ plan to 10 $ was super smooth by the way, so if you want to try for less $, the 5 $ for a month is enough so you can get used to the product and know if you like it or not (and that's besides the 100 free you get while signing up for a trial).
excellent, thank you for the information :)
That would be great. The $5 price point is the most I'd pay for search, but 300 searches per month is just not worth it in any sense. If all of the tiers moved down one step, or if Kagi offered a discount for annual membership, I could see it working for me.
I do 500-600 a month pretty regularly. They provide stats.
What happens if GOOG acquires Kagi?
I'm sure many folks, including me, would be willing to help Vlad organize a private acquisition if he suddenly had some pressing need to sell.
Don’t you work for Google or is that a different Michael Zuo?
I don't, but it's possible someone else does.
I signed up for HN before it was popular to include middle initials or other fancier forms for the username, I'll update my profile though.
I love the idea
I challenge you to find a group of rich folks who will fund such a thing - and will not simply demand the same profits as current structures do.
I just tried this with ~30 wealthy individuals/family funds that I have great relationships with and literally everyone said some form of the following:
“I am not going to liquidate my current investments that are on IPO trajectory for something that by definition will not IPO, and my investments are to ensure my kids go to private school/colllege fund/etc…”
You need to convince people, who currently think that the goal of making/having money is to insulate them and their families from reality, to instead choose to make/have slightly less money so that somebody else can have an easier/better experience in a way that is still concomitant with commercial transactions.
Huh, is this story a metaphor/allegory for something?
Why not 'challenge' yourself if you've already done quite a bit of groundwork?
Or at least the comment suggests you've already put more thought into this than me.
I’ve already put maximum financial and personal time inputs into my particular project, which is unfortunately insufficient to get started in earnest.
I need 200k to actually start what I want and currently have 5k in my bank account and a lot of mandatory overhead (kids etc).
If you’re legit interested I’ll tell you all the details 1:1
Sure we can talk, DM me on twitter.
I actually can’t…Twitter charges for DMs now apparently
Email me from the one in my profile
This is really the ideal use case for nonprofit funding, but unfortunately, real-world wealthy donors tend to give to university football programs and getting their names on hospital buildings. Sadly, SMU near me just made a deal with the ACC to forego ten years of television revenue in exchange for being allowed into the conference in order to get an automatic playoff berth if they can win the conference. Boosters will fully fund the athletic department to make up for the shortfall of not receiving and television revenue. All that money could have gone to good, but rich people care more about their alma mater having a 1% chance rather than a 0% chance at a national championship.
Happy customer here! Been paying for a year with no complaints. Amazing search engine -- kind of like how Google used to be years ago before they started injecting more ads into search.
I'm just trying Kagi out now, having done about 2 searches of my initial free 100. So far one was better than google and the other no worse. The "no worse" one was for something where I already suspect there aren't any good results to find.
Fingers crossed, but I have a good feeling about it. If it goes well the pricing seems fair.
I still try to use + but I guess that hasn't worked since before Google Plus. I never thought I'd want altavista back.
The solution is to simply stop using google and find other search engine, eventually decide to pay for really useful search engines like kagi.
I see that the quotes in your title are not regular ASCII quotes. I agree that Google's search result quality has been quite horrible, but could the fact that your system is somehow not emitting "real" quotes also be the cause? I've seen plenty of problems caused by those horrid "smart" quotes in the past.
A HN comment recently recommended Yandex describing it as what Google search used to be.
After seeing the post and trying Yandex it was absolutely right, it’s what Google search used to be.
Now whenever I use Google and it’s just a list of ecommerce adds or content farms duplicating the same content without substance, I head to Yandex and get the type of results I used to get from Google.
Agree. Yandex seems to perform better. Heavy use of captcha's (mullvad vpn), but once through that, enjoy the service.
Guess it's because Yandex gets most of its ad revenue from searches in Russian. Search results for English queries are not so ad-cluttered. For searches in Russian language, Google is still better (shows less ads per page), but tends to prefer .ua websites. I guess their ML predicts European IP + Russian language as a proxy for someone from Ukraine now, which is pretty logical.
- https://www.startpage.com/sp/search
... still supports it. Great search engine. Great story behind it.-
Wow I didn’t know they changed this. No wonder my quotes searches made no sense anymore…