Loading workgroup…
Workgroup
Implementing feedback mechanisms and reputation systems that reward positive contributions and maintain community standards.
A peek at recent conversation in DP18 - Feedback Loops and Reputation. Join to read the full history and post.
Join workgroup →Daveed's reply to Nada El Mais, sent by email:
Hi Nada,
Thank you for these notes. The line that stuck with me is your warning that we could end up building "a better comment box rather than a genuinely learning system." That, and your distinction between reputation as a score and reputation as evidence of contribution, gave me a clear way to sharpen the chapter.
I've turned your points into six proposals on DP18:
In DP18, below the sentence "Feedback is not engagement. It is a structured pathway from…", I've proposed: "That pathway is a closed learning loop: capture, interpret, decide, act, observe, learn, adapt. Each stage leaves a trace a participant can follow. The loop is closed only when the person who gave feedback can see that it was heard, what was done with it, and what changed as a result, including whether the change worked. A system that stops at capture is a better comment box, not a learning system."
In DP18, below the sentence "Reputation is not a universal score. It is contextual memory…", I've proposed: "This gives DP18 a design principle: reputation as evidence of contribution, not reputation as a score. A score is a verdict that hides its reasons, and once people know what produces it, they optimize for the number instead of the work. Evidence of contribution records who did what, when, in what role, and with what effect, and leaves interpretation to the community reading it. Replacing one engagement metric with a better-designed trust score would repeat the failure DP18 exists to prevent."
In DP18, below the sentence "Reputation should be portable where useful, but only with context…", I've proposed: "What travels should be the evidence of contribution, not a summary judgment of it. A person may be highly trusted in one community and little known in another. The receiving community should be able to read the underlying contribution history and interpret it against its own context and values, rather than inherit another zone's conclusion."
In DP18, below the sentence "Systems SHOULD make quiet, prosocial labor visible without forcing all…", I've proposed: "Reputation should recognize stewardship, not popularity. Moderators, reviewers, mentors, facilitators, knowledge curators, translators, documenters, and people who resolve conflicts often create the most value while generating the least engagement. Their work should register as distinct forms of standing, such as knowledge contribution, mentoring, governance, moderation, community building, or review, rather than depending on how visible or well-funded they are."
In DP18, below the sentence "Communities should periodically review whether feedback loops are actually improving…", I've proposed: "Feedback serves two purposes: evaluating individuals and helping the community learn. The second is often more valuable. Retrospectives should capture recurring lessons, decisions, corrections, successful practices, and unresolved issues as community memory, so that feedback does not disappear into an activity stream. DP22 (Civic Memory and Epistemic Continuity) governs how that memory is kept and resurfaced."
In DP18, below the sentence "A DP18-aligned implementation should be evaluated against the following questions.…", I've proposed: "One test sits above the detailed criteria. Over time, can the system answer three questions: Who has contributed meaningfully, when, and in what way? What has the community learned from those contributions? What changed because of that learning? A system that answers only the first is a reputation mechanism. A system that answers all three is community learning infrastructure."
Each proposal will show up in the chapter's discussion once it is posted, and you can comment on it or refine the wording there.
A few things I didn't turn into proposals. On contestability and decay, the chapter already covers this in some depth: 5.9 on decay, renewal and repair, 8.2 on inspecting and challenging attestations, and the non-goal that rules out permanent records as punishment. Rather than repeat it, I've leaned on that existing text, but tell me if you think something is still missing. Your requests for visualizations of the loops and relationships, for weaving the examples into the prose rather than leaving them as points, and for rethinking the chapter structure are all fair. They are bigger editorial changes than a single sentence proposal, and with the V1 freeze on November 9 I need to decide what is feasible for this version and what goes into V2. I'll follow up with you on that.
Thank you as well for the Keen example. It is exactly the kind of case DP18 should protect: real impact with no budget for visibility.
With thanks,
Daveed
Zanette Johnson sent these notes on DP18 by email:
My feedback comes from the perspective of a non-technical reader. To understand this content well, I'd need three things: an opening summary, a glossary, and probably way more examples!
Reading through, I noticed I was constantly needing to search for definitions of terms on the web and in other parts of the book, and it was hard to find context for them (decay schedule, zone scope, rollback triggers, contributor recognition gaps, ML-RFC, etc.).
I'd move the core principle section up higher and turn it into a summary of the chapter/desired feature before getting into the technical parts, so that people can JUST read the summary. Most likely, all the chapters could do with summaries — then you could compile them as an additional asset for broader audiences to grok what the Meta-Layer is all about.
There was repetition throughout Sections 4, 5, 8, 11, 14, and 15 that I'd check for. They seem to restate things in different ways, which makes it even more confusing. The failure modes overlap too, especially between Section 3 and the "Failure mode" lines that close each subsection of Section 5, where several seem to name the same problem twice. This may very well be due to my lack of understanding so take it or toss it if you see a need for that redundancy to disambiguate things.
I love Sections 12 and 13-- that's where things finally came together for me in a picture of DP18 as a whole. "Oh yes, reputation domains! Relationship to other DPs!" That's the point where I felt like: Finally I'm getting a picture of what we're doing here instead of floating around in what we're NOT doing.
The questions in Section 16 are really important, and it's excellent to see them all in one place. For me to follow the path toward ML-RFC in Section 17 requires more context than I've got, but Section 18 lands it with meaning.
So, no in-line edits for me, but I hope my comments are helpful in some way!
Daveed's reply to Zanette Johnson, sent by email:
Hi Zanette,
Thank you, truly. You've been in my corner on this work for a long time, and it shows in how carefully you read. Your note that you kept leaving the chapter to look up terms, and that Sections 12 and 13 were where you finally saw DP18 as a whole instead of "floating around in what we're NOT doing," is exactly the kind of reading I can't do for myself anymore.
You said no in-line edits, so I turned your comments into five proposals myself, credited to you:
In DP18, above the sentence "This draft articulates Desirable Property 18 (DP18) as the condition…", I've proposed: "In brief: DP18 is about how a community listens and how it remembers. When people speak up, they should be able to see what happened to what they said. When someone does good work or causes harm, the community should remember it in the place where it happened, without turning it into a single score that follows them everywhere. And people should always be able to question that memory, repair their mistakes, and grow past them."
In DP18, below the sentence "This prevents reputation from becoming a single global score while…", I've proposed: "Two of these terms in plain words. Zone scope is the community or space where a reputation counts: standing earned in one zone applies there, not everywhere. A decay schedule is the agreed timetable by which older signals count for less unless recent activity renews them. For example, someone known as a careful reviewer in a neighborhood gardening zone carries that standing in that zone, not in a medical advice zone, and a review they did three years ago counts for less than one they did last month."
In DP18, above the sentence "Dashboards must be designed to support learning, not surveillance or…", I've proposed: "Two of these items need a plain explanation. A contributor recognition gap is work the community depends on that nobody is credited for, such as the person who welcomes newcomers and answers their questions every week. A rollback trigger is a warning sign agreed in advance that, if it appears, means a change gets reversed, such as a new posting rule that doubles the number of moderation appeals within a month."
In DP18, above the sentence "The goal is neither amnesia nor permanent stigma. The goal…", I've proposed: "For example, a participant shares a health claim that turns out to be wrong, and a steward marks it as inaccurate. The participant posts a correction. Over the following months their contributions are accurate, the old mark counts for less and less, and after a set period it no longer affects what they can do in the zone. If they had instead caused serious harm that was never addressed, that record would stay visible until it was resolved."
In DP18, below the sentence "DP18 is currently an ML-Draft and serves as exploratory scaffolding…", I've proposed: "In plain terms, an ML-Draft is a working proposal: it sets out ideas for communities to test and argue about. An ML-RFC is the more settled stage, reached when communities have agreed on shared formats and seen them work in practice. The naming borrows from the internet's long-standing Request for Comments documents. The lists below describe what DP18 needs before it can make that move."
Each of these will show up in the chapter's discussion once it is posted, and you can comment on or refine any of them there. If any definition still doesn't land for you, that's the most useful thing you could tell me.
What I didn't turn into proposals, and why:
A glossary for the whole book, and summaries for every chapter compiled into a separate piece for broader audiences, are both great ideas. They're book-wide decisions rather than a sentence in DP18, so I'm taking them up separately. The In brief proposal above is a first try at what a chapter summary could look like.
Moving the core principle section up is a structural change. With the V1 freeze on November 9, I'd rather add the short summary at the top now and look at reordering for V2.
On the repetition across Sections 4, 5, 8, 11, 14 and 15, and the failure modes in Section 3 overlapping with the "Failure mode" lines in Section 5: you're not misreading it. Some of that is deliberate, since Section 3 describes what goes wrong in the world and Section 5 names what each mechanism guards against, but some of it is plain redundancy. That needs a careful editing pass rather than a single proposal, and I'll follow up with you on it.
And thank you for the kind words about Sections 16 and 18. I'm glad the open questions read as important rather than unfinished.
With thanks,
Daveed
Ramina sent these notes on DP18 by email:
I took a quick look at the chapter you sent. Overall speaking, I really like the accountability and idea of reputation being tied to context... A few thoughts:
- I feel it’s pretty long and repeats itself. Sections 5, 11 and 15 cover a lot of the same things. A short main section with the details at the end would be easier to read...
- What about unpopular/controversial opinions? Say someone shares an opinion that might be true but most people disagree with. I couldn’t find how they’d be protected from downvotes, mass reports or losing reputation. It also isn’t clear who decides what’s “accurate”, or how disagreement is told apart from actual harm.
- If we publish how reputation is calculated, people can game it. We want people to be accountable but also let them report anonymously. We want people to be able to move on from mistakes but also keep serious harm visible. It would help to say how these get balanced.
- Receipts, stewards, appeals and audits are a lot for a small group to run. Maybe there could be a lighter version for small communities...
Daveed's reply to Ramina, sent by email:
Hi Ramina,
Three of your points are now proposals, credited to you:
In DP18, below the sentence "A verified expert correction, a peer endorsement, an anonymous…", I've proposed: "Unpopular is not the same as wrong. A view that most participants disagree with must not lose reputation or visibility through downvotes or mass reports alone; reputation responds to evidence of harm or inaccuracy, judged through a stated process with appeal, not to disagreement. Each zone says who decides what counts as accurate, and on what evidence."
In DP18, below the sentence "Systems SHOULD prevent AI from flattening minority concerns into…", I've proposed: "Some goals in this chapter pull against each other, and each zone states how it balances them: publishing how reputation is calculated against making it easy to game; accountability against the ability to report anonymously; letting people move on from mistakes against keeping serious harm visible."
In DP18, below the sentence "They are non-normative and are offered as an assessment aid rather…", I've proposed: "Receipts, stewards, appeals, and audits are a lot for a small group to run. A lighter profile for small communities keeps the essentials, which are visible feedback routing, scoped reputation, decay, and a way to appeal, and lets the rest be added as the community grows."
They will show up in the chapter's discussion, and you can comment on or refine them there.
On length and repetition across sections 5, 11 and 15: I agree, and a restructure like the one you describe is a job for the DP18 workgroup rather than a single proposal. I am passing that on.
With thanks, Daveed
Community Chats match workgroup Collab: human messages in the main panel and private Deepi in the right sidebar. Invite people outside the workgroup by email or link. This is separate from Workgroup Chat, which stays inside the member thread.
Start a Community Chat for DP18 - Feedback Loops and Reputation. You pick who joins. Everyone invited can post in the member chat and use private Deepi in the sidebar. The chat appears in your Deepi sidebar with a Community badge.
Sign in to start a Community Chat from Deepi.
Opens Deepi with the invite flow. Community Chats also appear under My conversations at /agent.