Why is duckduckgo suddenly being marked as Highly Mature?

  • 3
  • Problem
  • Updated 4 years ago
I've been using the duckduckgo.com search engine for many years. Covenant Eyes has never complained about this on our accountability report. But in the last week or so, duckduckgo has suddenly become a Highly Mature site. This is a problem because our reports are now repeatedly being flagged as "needing review" despite having no problematic content. Too many false positives means that legitimate concerns can be hidden in the "noise."

(I use DDG because it hides my search activities from the Google/Yahoo/Microsofts of the world. There is no reason for these corporations, or the governments under which they operate, to have easy access to my online behavior.)

I've tried adding duckduckgo.com to the Allowed list on the filter, but this does not seem to have any effect on the accountability report.

How can I keep the legitimate use of DDG from rendering the accountability report useless?
Photo of TC

TC

  • 9 Posts
  • 4 Reply Likes

Posted 5 years ago

  • 3
Photo of John

John, Official Rep

  • 439 Posts
  • 79 Reply Likes
DDG is an HTTPs search engine, hence the reason it "hides" your information. (Truth to be told all it can really hide is who is searching, but that is off topic)

Seeing as you are using Covenant Eyes to help protect you or your family online, HTTPS sites can be problematic.  Depending on which version of Covenant Eyes you are running, and within what browser, it is possible that searches done on a site like DDG would not be rated properly, if at all.

My understanding is that we have rated HTTPS sites like DDG, that we may have trouble monitoring, higher in order call attention to their use. The HM rating in this case would be more to call attention to the fact that DDG can be used to potentially hide searches from an accountability partner.

Let me have a chat with our rating team and see if this is a recent change. I will post back here when I know more.

Thanks for taking the time to post
Photo of TC

TC

  • 9 Posts
  • 4 Reply Likes
Any update on this? DDG has a Settings link that allows me to disable
the use of https. (I also needed to remove duckduckgo.com from my HTTPS Everywhere Firefox extension.) Now I have DDG operating strictly from http, but the detailed browsing log still shows it as "HM".
Photo of Cory M. Crockett

Cory M. Crockett

  • 11 Posts
  • 0 Reply Likes
As A Frequent DEC User, I Would Appreciate You Keeping Me Updated On Any Developments Relating To This Topic.
Photo of Annelise

Annelise, Official Rep

  • 257 Posts
  • 13 Reply Likes
Hi Cory,

Thank you for your inquiry about Duck Duck Go! We have recently changed the Duck Duck Go rating from Highly Mature to Everyone. We are now able to see what searches are being done in Duck Duck Go, however the rating can fluctuate based on what is being searched for in the Duck Duck Go search engine. This rating change will prevent any inaccurate flags on your report concerning Duck Duck Go. 

If you have any further questions do not hesitate to call our Customer Service at 877.479.1119.

Best regards,
Annelise
Photo of TC

TC

  • 9 Posts
  • 4 Reply Likes
Thanks for your quick reply, John.

Out of curiosity, I did some closer looking at the detailed browsing log and noticed that DDG is actually rated "E" in those. But on the reports they still come up as "HM." These are for identical time periods. Wonder if that's a clue?

And, yes, "anonymizes" would have been a better word than "hides". I do understand and appreciate the reasoning behind CE flagging some https requests. I guess what I am asking for--assuming the rating continues to be HM--is a whitelist feature like the Allow list for the filter function.
Photo of sonic1100

sonic1100

  • 4 Posts
  • 0 Reply Likes
I second this. Every single DDG search gets flagged, and all the false positives make a genuine review challenging. Please come up with a work-around for this!
Photo of TC

TC

  • 9 Posts
  • 4 Reply Likes
Still waiting for an update... If HTTPS isn't the problem with DDG, what is?
Photo of John

John, Official Rep

  • 439 Posts
  • 79 Reply Likes
I have not heard anything back on this issue yet. I will ping our rating team again to see if I can provide a better explanation (our rating system is not something I have loads of expertise with).

To some extent DDG was designed to hide your data, and that (in a way) will always conflict with the mission of Covenant Eyes. We want to deliver a solution that will allow you to have the best of both worlds, but we will not release it until we are sure that it does not allow members to use sites like DDG to simply bypass everything that we stand for
Photo of Chad

Chad, Alum

  • 122 Posts
  • 10 Reply Likes
Without waiting for a response from the rating team, John, I'll jump in with a response to this issue.


In the plainest sense, search engines like duckduckgo hide a users searches and internet activity, in our experience this is used primarily for the search of pornography. While duckduckgo may not advertise their search engine for that use, it has become the primary use of search engines that provide anonymity.


As John alluded to eariler in the conversation, if you are using Covenant Eyes to bring transparency to your internet activity then a search engine like duckduckgo is not a wise choice.


Long story short and without getting into technical details, use a different search engine.


Chad
(Edited)
Photo of sonic1100

sonic1100

  • 4 Posts
  • 0 Reply Likes
This is a distinctly unhelpful and unsatisfying response.

I understand you to make, directly or indirectly, the following points: 1) DDG is frequently used for porn searches; 2) DDG somehow cloaks/tries to cloak its users searches from CE; and 3) if you use DDG, you are probably trying to evade CE.  

Point 1 may be true, but seems totally irrelevant. Similarly, Point 3 appears to have little to do with CE's technical ability or inability to stop flagging every DDG search, and is inappropriately dismissive of the fact that there are some very compelling, legitimate reasons why people might want to use DDG (such as not wanting to have every search query tracked, profiled, and monetized by a increasingly creepy purportedly-beneficent internet overlord).

Point 2 seems to be the only point that could potentially bear the load of argument. If it is indeed true that DDG's anonymity features can somehow stymie CE's monitoring ability, allowing CE users to search for inappropriate material with impunity, then I can understand why it would be marked as highly mature. But if this is true, how can CE currently see everything I search for on DDG?  Is it because there is some disconnect between CE's keystroke logging, and its link-based analysis? Does CE actually base its filtering entirely on link analysis, rather than on a simple text analysis? If DDG prevents CE from analyzing content, it seems like it would be a fairly straightforward matter to record all text entered on DDG and cross-reference it against a database of inappropriate words or phrases instead.

My question thus seems to boil down to the following: if CE can currently detect, record, and flag every search query on DDG, despite ongoing "cloaking" features, then why can't CE detect, record, and flag only inappropriate DDG queries?

Also, you completely ignore TC's point. At the very least, CE could provide an option to whitelist DDG, notify accountability partners to that effect, and allow its users to make that choice.

If you don't know the answer to these questions, then say that, and find someone who can. Don't trot out dismissive platitudes implying that CE subscribers who use DDG users are trying to circumvent CE's transparency functions.
(Edited)
Photo of Chad

Chad, Alum

  • 122 Posts
  • 10 Reply Likes
Thank you for your comment sonic1100, I can tell you have thought about this quite a bit.

The answer is really quite simple. Covenant Eyes is a tool that allows people to maintain their integrity while using the internet and to protect the ones they love from dangers and temptations that can be found online.

CE, like any tool must be wielded properly, afterall a hammer can't drive a nail itself. To use CE properly it requires the user to want to be held accountable, to want transparency in their internet use, to have what can be secret brought into the light.

We speak with members every day who want these things and are willing to put up with the minor technical issues that can arise from the complex process of monitoring, recording and accurately scoring the URLs your internet capable devices download.

Rest assured that the team here at CE work hard to overcome the technical issues that arise and to stay apace with the ever-changing landscape of the internet. I do sincerely apologize that duckduckgo is currently rated Highly Mature, but really, any anonymizer should be.

However, I still maintain, that if any member of CE truly desires real accountability then simply using a different search engine until such time as we develop a sound method to accurately monitor https traffic should not be too much of a sacrifice. This IS an issue we are working on, but we do not want to jump the gun and release an incomplete solution.

Chad
(Edited)
Photo of sonic1100

sonic1100

  • 4 Posts
  • 0 Reply Likes
Thank you for your reply.  I appreciate the need to develop a complete solution, and I am willing to use a different search engine for a while if need be.  My point is simply that DDG is NOT currently hiding anything from CE.  Every search query is captured by CE's monitoring function (and marked as Highly Mature). Therefore using it is not an effective means of avoiding accountability.

If for some unknown reason, the situation is binary - either CE captures every query and marks every visit as Highly Mature, or CE does not mark the site as Highly Mature and captures nothing - then I can understand why things are the way they are. But it seems more likely to me that it isn't an either/or situation, and that it is possible to capture all queries, but mark less than all of them as HM.  That's my only point.
Photo of TC

TC

  • 9 Posts
  • 4 Reply Likes
I appreciate the responses and support the overall goal of CE preventing the hiding of activities from the accountability reporting. However, I remain unsatisfied and echo sonic1100's frustration.

DDG is about anonymizing the interaction with the search engine. It does nothing to hide traffic from the client. When DDG is used over HTTP, traffic in both directions is subject to the same scrutiny as traffic to Google or Bing. If CE can flag inappropriate requests for these other search engines without flagging every usage, why can this not be done for DDG? I get the point about HTTPS, but DDG can be used without HTTPS and should not be marked "HM" in this case.

Out of curiosity, I searched both Google and Bing with HTTPS. CE marked these activities as "E", despite the fact that they could presumably be used to hide a user's activities more effectively than DDG with HTTP.

I don't understand.

Thanks for taking this seriously, and I look forward to seeing the in-progress solution that Chad alludes to.
Photo of eggzandham

eggzandham

  • 9 Posts
  • 2 Reply Likes
Hi, I'd like to voice my frustration with this as well. I've even talked to someone on the Rating Team, and her response didn't fully make sense, either. Her main concern, stated over and over, is that CE doesn't know if/when a user is performing an image search. Even though that's true on the URL level (Google uses the "images" prefix in the url, whereas DDG does not), that's NOT true on so many other levels.

Even if this WERE true, my main beef is that this isn't consistent with other https sites. I can guarantee you that searching for "cat" on DDG is much safer than using the same search on, say, Youtube or Vimeo. Yet out-of-the-box, Youtube and Vimeo -- both on https -- come with lower ratings.

Even going down a few notches with DDG would be a huge improvement (like default to MT or something). I even suggested downplaying this url (ie, mark the following url as something lower than HM):

https://safe.duckduckgo.com/

-- but to no avail. 

DDG is becoming more and more popular -- now an available choice for Safari/Mobile Safari for Yosemite/iOS8, as well as for Firefox. If it's labeled as HM, (1) there needs to be a good reason, and (2) CE needs to maintain consistency across all https sites. Right now, CE satisfies neither (no GOOD reason and not consistent), which makes this case particularly frustrating.
Photo of Patrick Smith

Patrick Smith, Alum

  • 147 Posts
  • 21 Reply Likes
Hey, all. I thought I'd chime in on this thread briefly to shed some light on why the rating for DDG hasn't changed...yet.

In the last year or so, we've made some pretty substantial changes to our Windows and Mac clients. Among those changes is the ability to "see" the full path of an HTTPS URL. Previous clients were unable to see beyond the domain e.g.https://google.com/?gws_rd=ssl#safe=active&q=puppy+dog   vs.  https://google.com

From what most of you guys on this thread are describing, it sounds like you're running a newer software client (i.e. even on a secure site, we're seeing and reporting beyond the domain). That's why you're mentioning that you can see the search terms in the URL. If you were on an older client, you'd only see https://duckduckgo.com .

Theoretically, if all of our customers were on newer clients (and we could see & score the entire path), we could allow duckduckgo.com to run through our scoring system organically. Unfortunately, we still have tens of thousands of people running older clients. For them, we can only see the domain, and have no ability to report on what was searched for. We have one scoring system serving both old and newer clients, so we don't have the ability to score duckduckgo.com one way for newer clients and another way for older ones. 

We're in a tight spot on this one. On one hand, we must fulfill our ethical obligation to alert members and their accountability partners to potentially objectionable Internet use. On the other hand, we want to respect the common desire for privacy among our members. I am sure you see the tension there. In keeping with that ethical obligation, we've decided to continue scoring sites like duckduckgo.com highly until we reach adequate uptake of our newer clients. We're actively pushing upgrades etc., and we're making good progress. 

I don't presently have a time estimate for you; I wish I did. Until then, it seems like your options are to drive your accountability partner crazy with a boatload of DDG links on the report or to use another search engine.  

I know this is a frustration, and it's one with which I genuinely sympathize. I continue to prioritize work for our development teams that will yield the most favorable results for the highest percentage of our members. I don't always get it right, and even when I do it isn't always super speedy. 

The software--both front end and back end--continues to improve, though. I think we're headed in the right direction as we work to provide tools that encourage accountability and trust in the fight against Internet temptation. 

In the meantime, I appreciate your patience. I hope this sheds a little light on things. Sometimes just knowing the rationale is helpful. Thank you for your willingness to chime in here, and please keep us apprised of your experiences.
Photo of eggzandham

eggzandham

  • 9 Posts
  • 2 Reply Likes
Hi Patrick, thanks for the feedback. Though this doesn't solve the main issue at hand, like you said, just knowing the reasoning behind things is helpful -- and I think this is the first explanation that makes the most sense to me.

I do hope this gets expedited, as I now currently work there, and it is driving my accountability partner crazy... =\

HOWEVER, would you guys make an exception for safe.duckduckgo.com ?
(Edited)
Photo of sonic1100

sonic1100

  • 4 Posts
  • 0 Reply Likes
Thanks so much for weighing in, Patrick!  Like Abe said, at least I can understand what's going on now.  I really appreciate the update and look forward to more progress!
Photo of TC

TC

  • 9 Posts
  • 4 Reply Likes
Thank you, Patrick, for your comments. It is helpful to know the full reasoning behind the decision, and it is encouraging to know that it will be improved.

Is it correct to say that CE does not distinguish between http and https when rating sites? I assumed that CE would have distinguished between these when accessing DDG, but it sounds like you're saying that the only criteria is the domain name and path. I'm running version 5.0.4.280 of CE, and I haven't been able to see search terms when using DDG over https. However, I do when it connects over http, and this is why I was surprised that CE would still mark it as HM. But if the domain name and path are all that are considered, this makes more sense to me.

In any case, I'd still echo Abe's request to have safe.duckduckgo.com rated a little lower.
Photo of eggzandham

eggzandham

  • 9 Posts
  • 2 Reply Likes
Patrick, it occurred to me that if this is true for DDG, then isn't this also true for Google (and anything using https)? More and more sites force you to use https (vimeo, flickr, to name a few), so in order to remain consistent, shouldn't THOSE sites also be labeled HM? And if not, then what's the distinction?
Photo of Munsell Karen

Munsell Karen

  • 1 Post
  • 0 Reply Likes
Very interesting thread and great conversation. Love that viewpoints can be shared here with a common goal of accountability without compromise but functionality that works for most. I stumbled on this thread but it was timely for an issue I had noticed just today that was exactly the same. It is with startpage.com though which was a search engine specifically developed to prevent intrusion but in effect works against covenant eyes so for now we will not be using it until this gets resolved. Its good to know why it doesn't work though I figured it was something related to the "secrecy" of its searches. Thanks again for listening and continuing to work on this very important issue of internet accountability.
Karen
Photo of tiapa

tiapa

  • 4 Posts
  • 0 Reply Likes
I'm glad to see DuckDuckGO  (DDG) finally working. Yea! Drove me nuts for quite a while.  I utilize DDG when I need accurate, usually technical,  results without those results being skewed by my location, prior search habits, my buying habits, computer type etc.  I'm so glad it is back.  

I too was baffled that CE could see everything I searched for in DDG and every site I went to prior to DDG being nixed.  I still have not heard a valid explanation about DDG as well as HTTPS and SSL.  

In the future when CE believes it is unable to rate a site properly and wants to block it PLEASE do not use HM as the rating !!!  Many of us rely on accountability to maintain our freedom.  To try and explain massive amounts of false positives HM ratings to partners and officials who don't want to hear it is nerve wracking.  Recently I received about 1472 HM ratings for clients2.google.com (Google Search Screen) and 3 HM's on my own router address.  The assumption from those who read my reports is "so what did you do to cause this?".

In an email from the CE "rating change" group I read the following: "Our system automatically defaults to a high rating and for Filter users, it gets blocked.".   I was shocked!  Don't do that!  For some of us that is the same as being falsely accused.  If CE cannot rate a site due to program limitations, again, please do not rate it as HM.  Man up and rate it as "UK" (for unknown) or something, even if you block it.  Don't let the explaining fall on us.

With all that being said I love Covenant Eyes.  I never want to be without it.  There is no better software for accountability and blocking. I love the CE crew, consummate professionals.  I don't worry much about misleading title to links, they get blocked.  I am a very happy user and quite often recommend CE.

Photo of John

John, Official Rep

  • 439 Posts
  • 79 Reply Likes
Tiapa,

Going to jump in here and respond to this excellent post. 
I too was baffled that CE could see everything I searched for in DDG and every site I went to prior to DDG being nixed.  I still have not heard a valid explanation about DDG as well as HTTPS and SSL.  
HTTPs and SSL are a tricky business in the Accountability/Filtering industry. If we provide too much information we allow members to begin working towards circumventing CE protection, but if we do not provide enough, we create a system that feels arbitrarily punishing. Neither of these is good (obviously) but we tend to err on the side of caution when it comes to our members integrity online. 

That being said, here is the fundamental problems with sites that have HTTPS encryption:

HTTPS is designed to encrypt any personal data that you exchange within a website. Traditional means of getting information from these sites then (URL sniffing, packet monitoring etc) are rendered somewhat useless. It matters not if we could get information, when the information itself is encrypted in such a way as to be rendered impossible to report on. 

There are common ways of "breaking" HTTPS (a Google search will reveal most of them) but they come with some interesting costs that make them less attractive to legitimate companies. (Spyware would not care, for instance) 

So the pain point for CE right now is how to "break" HTTPS in a way that both provides the accountability that is needed by our members, without exposing them to security risks or revealing information that their partner does not need to know. We have some cools steps in that direction coming, but they are not foolproof yet. We understand the gravity of the situation though and it is something that we are actively working on.

In the future when CE believes it is unable to rate a site properly and wants to block it PLEASE do not use HM as the rating !!! 
This is a fantastic point. I actually took this exact post and wording up the chain to spark a conversation about what it means when we rate a site as HM. The general consensus so far is that we need to better differentiate between unrateable sites and HM sites.  Obvious, right?

Well maybe not... I am not denying that the current rating system is not providing enough clarity for the user. (I am in support of change) but there is something worthwhile about the history of using HM for unrateable sites.

 See back when CE was started, there were extremely few sites using any sort of subterfuge methodology, (modern examples would be HTTPs, Proxy, VPN, etc) so any site that we could not rate was quite likely from a user trying to bypass our product. That has certainly changed in today's internet age, but the value in being stern about reporting attempts to bypass our software remains.

Thanks for taking the time to write out your thoughts, and we really appreciate your passion. I am very pleased that conversations like this can develop organically in this environment.

thanks,