Duckduckgo browser

cregox · October 20, 2020, 11:54pm

they give this weird “unknown privacy practices” for virtually every website. which brings a “B+” rating at best.

i could only see their own domains being rated as “A” and getting “good privacy practices”.

Screenshot_20201021-005031

in either case, they link to you.

and i see there’s a rather manual way to add a website there, by requesting the service to be added:

then again, none of the ones i took as sample from that list gets the actual “promised” badge there.

well, not even tosdr own website gets and A!

any clue on what’s going on?

Dr_Jeff · October 21, 2020, 1:07am

The service points reviewing process is slow, since it’s mostly manual and volunteer based. We’re striving both to review as many as we can, and to eventually automate the process, but the latter needs a lot of code tampering and inquisitive structuring, which isn’t easily achievable since this is an OpenSource project, and the scale is huge.

We upload extensively reviewed Services (sites) to our database by default, since uploading incomplete reviews would not be good IMO.

Regarding the DuckDuckGo extension, when a site review hasn’t been uploaded to our database, the extension sets its rating to B+ by default, but shouldn’t necessarily represent our official rating. For that we have a browser extension.

cregox · October 21, 2020, 6:07am

thank you for the reply! and for making this open source!!

sorry, i don’t have a desktop to see the extensions.

(on a side note: personally, i would love helping to automate the process, if that makes any sense… but this would become a much more personal talk). anyway…

still, i don’t quite understand what’s going on.

ok, so when the ddg android browser points to tosdr for “good practices” and then going to the tosdr webpage with this browser doesn’t bring “good practices”, it was already very clear something is broken.

now, from what you just told me, looks like the issue here is on the tosdr manual process to feed “the database”, rather than how ddg polls the data from there (or this upload process), and it even sounds the ratings should ideally always be the same…

but then, the ddg browser promotes most websites to B or B+ “by default” (when the privacy practices are unknown) apparently at random, since wikipedia gets 0 tracker and B (which agrees with tosdr apparently also at random), but medium gets 3 trackers blocked and promoted from D to B+ (while tosdr shows “no class yet”).

it’s also very easy to see how tosdr ratings, at the very least, don’t match on google (D vs C) and facebook (C+ vs E) when comparing it with your website. so looks like you’re not using the same database unless the website isn’t the official and only the extension is.

of course, my curiosity also comes from the interest of getting a nice rating on my own websites (which i usually encourage everyone to read, no direct link here to avoid looking like spam), but with all those issues i don’t see much hope for the near future.

to hopefully make it easier, here are a few more direct questions:

why tosdr have no official review by tosdr?
why is tosdr not with an A on ddg? isn’t there a partnership?
why the ratings differ?

it still feels i’m forgetting something…

anyway, on the side:

how exactly do you think i could personally help?

because i already know that most people can help with the manual process (i probably won’t), some with getting the hands in the code (not sure i can), and others with donations or otherwise financial (for sure i can’t).

ps: unproud time taken to write this post was 1h

KnossosDomovoi · October 21, 2020, 1:13pm

From what you describe, this sounds to me like a problem with the DuckDuckGo Privacy Browser, instead of an issue with tos;dr. You claim that they promise to use the tos;dr ranking, but they don’t. I suggest that you open a ticket on their issue tracker so they can fix it.

cregox · October 21, 2020, 1:21pm

let’s see how it goes.

that would answer 1 of the questions…

Peepo · October 23, 2020, 10:53pm

It looks like someone from DDG responded to your issue on Github, but it is their app not updating to our website. DDG last updated the information from our website approximately 15 months ago, which is why the grades on their app differs from ours. There is a partnership with us and DDG but whatever grades were on our site at that time is still there. I emailed DDG about it and Michiel made an issue on Github about it a couple of months ago https://github.com/duckduckgo/duckduckgo-privacy-extension/issues/482.

Another thing is that edit.tosdr.org has the most current grades than the front page of tosdr.org (DDG is still behind on our front page though). This is because we have to manually crawl edit.tosdr.org and it hasn’t been crawled lately. There isn’t a reason why ToS;DR doesn’t have an official review and has a grade on the front page, I will try to fix that. Also, I’ll message you about helping!

cregox · October 24, 2020, 11:36am

yes, but their answer there was regular “bad”.

yours on the other hand, spot on! thanks for clarifying everything!!

solved!

now…

sorry for going more offtopic here, but this is one big problem with forums… they don’t make it easy to brainstorm:

some discourse forums offer the “mark as solution”, which is what i would have pressed there.
having a manual crawling process for 2 instances of your own domain sounds like terrible engineering choices. (i don’t have any answer for how to make this better in specific, but for sure i could quickly find about it if by any chance you need to and if i’m guessing the right scenario here.)
i couldn’t find you on the github… are you the privacy philanthropist? if so, i would love to hear what you think of my ideas around privacy!
how many people are there in the team?!
any suggestions for a better way to do storming with tosdr?

justin · October 24, 2020, 5:28pm

I am planning to install plugins soon once @michielbdejong gave me access ^^

About Us -- Terms of Service; Didn't Read + 2 People who are not on the list yet. So about 12 of whom 4 are active

@michielbdejong, @Peepo, @Dr_Jeff and me

As for the DDG Privacy Extension they do their own Rating as it seems. I’ve skimmed through their GitHub and they pull the latest cases live from our API however their list of points is quite outdated and they do their own rating.

github.com

duckduckgo/duckduckgo-privacy-extension/blob/main/scripts/tosdr-topics.json

{
    "bad": [
        "they can license to third parties",
        "personal data is given to third parties",
        "may sell your data in merger",
        "targeted third-party advertising",
        "tracks you on other websites",
        "logs are kept forever",
        "they can use your content for all their existing and future services",
        "your data is used for many purposes"
    ],
    "good": [
        "personal data is not sold",
        "you can request access and deletion of personal data",
        "no third-party access without a warrant",
        "no tracking"
    ]
}

github.com

duckduckgo/duckduckgo-privacy-extension/blob/8dd747d0ed8fae9ee0158c7dea07368ab9827d52/scripts/tosdr.js#L16


      
          const request = require('request')
          const topics = require('./tosdr-topics.json')
          const fs = require('fs')
          const tldts = require('tldts')
          let processed = {}
          let nProcessed = 0
          
          function getSites() {
              // get the full list of tosdr sites. This does not include points data. We will
              // have to make a separate request for that.
              request.get('https://tosdr.org/index/services.json', (err, res, body) => {
                      let sites = Object.keys(JSON.parse(body))
          
                      // recurse through sites list. Get and process the detailed points data for each
                      getSitePoints(sites).then(result => {
                          fs.writeFile('shared/data/tosdr.json', JSON.stringify(processed, null, 4), err => { if(err) console.log(err)} )
                      })
              })
          }
          
          function getSitePoints (sites) {

github.com

duckduckgo/duckduckgo-privacy-extension/blob/8dd747d0ed8fae9ee0158c7dea07368ab9827d52/scripts/tosdr.js#L36-L37


      
          let githubRepo = 'https://raw.githubusercontent.com/tosdr/tosdr.org/master'
          let url = `${githubRepo}/api/1/service/${site}.json`

Basically the workflow of the API is:

Phoenix → ToS;DR Public API → DDG

And the current Public API has been updated 18 days ago. IIRC michiel wanted to do a cron for this

Peepo · October 26, 2020, 12:35pm

I am the privacy philanthropist! The only thing I’ve done on GitHub is upload icons, I don’t know how to program. You bring up interesting points in your piece but I don’t know if I agree that “privacy is mostly to secure lives from violence”. It can prevent violence and possibly start it, but I don’t think that is its main goal. Also, I disagree that “science only exists because we decided to trust our memory”. I understand where you’re coming from, as science is more of a concrete thing but maybe something like writing would better replace science in this case but those are my opinions

cregox · October 26, 2020, 4:08pm

thanks for all the extra information and digging there, @justin!

and thanks for the awesome feedback @Peepo!

there’s much i need to fix there still.

do you think it’s fine to hijack this “closed” topic and go sideways here?

if you want to edit and move posts around to invite more people into this discussion, be my guest.

privacy versus violence

as usual, wikipedia explains it perfectly:

Privacy is the ability of an individual or group to seclude themselves or information about themselves, and thereby express themselves selectively.

whenever i talk about privacy with people for long enough, in the end, it always boils down to violence prevention in this exact sense: to be free.

but even wikipedia can’t always explain concepts so succinctly:

Violence is “the use of physical force so as to injure, abuse, damage, or destroy”.[2] Other definitions are also used, such as the World Health Organization’s definition of violence as “the intentional use of physical force or power, threatened[3] or actual, against oneself, another person, or against a group or community, which either results in or has a high likelihood of resulting in injury, death, psychological harm, maldevelopment, or deprivation.”[4]

i like to think about it simply as intentional abuse of power.

fair enough?

perhaps there’s some better choices of word there, or perhaps i could simply include those definitions there… but i still think the concept there is very solid, even if it may sound disconnected.

science versus memory

that was a much bigger stretch indeed… i’ll probably choose better words there sooner now that you pointed it out. thanks!

but…

i have no idea what you meant about “something like writing would better replace science”.

i’ve written that science exists because we decided to not trust our memory.

in other words, realising our memory is very flawed was one of the biggest reasons that allowed science to become what it is.

coding

do you want to learn?

Peepo · October 26, 2020, 6:26pm

To keep categories on here focused, I’ll message you my response

justin · February 1, 2021, 5:00pm

This topic was automatically closed after 38 hours. New replies are no longer allowed.