Compare commits

..
67 Commits
Author SHA1 Message Date
Kristian WindandClaude Opus 5 1437667483 fix(notes): stjerne og arkiv står oven på hinanden
Release / Build and publish image (push) Failing after 9m56s
To knapper ved siden af hinanden er toogfyrre pixels taget fra en liste, der er
tre hundrede bred — og de er det, man rører en gang imellem, mens titlen og
uddraget er det, man læser hele tiden. I en søjle koster de toogtyve.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-20 11:23:21 +02:00
Kristian WindandClaude Opus 5 035d079a8e fix(notes): teksten fulgte ikke med, når ruden blev bred
Release / Build and publish image (push) Failing after 9m59s
46rem var sat for at holde linjelængden i skak, og det er stadig det rigtige
hensyn — men det blev sat som ét tal, og på en skærm halvanden gang så bred stod
teksten i en smal søjle med en håndflade tom plads ved siden af. Kodeblokke og
billeder, som er de to ting i en note, der *vil* have bredde, blev klemt sammen
uden grund.

clamp frem for et tal: en smal rude får det hele, en bred får en linjelængde,
øjet kan finde tilbage i. På 1920 er tekstsøjlen nu 1088 px af arkets 1304 i
stedet for 736.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-20 11:08:26 +02:00
Kristian WindandClaude Opus 5 aadfb197b6 feat(notes): arkiv, flere ad gangen, og en kopiér-knap på koden
Release / Build and publish image (push) Failing after 9m55s
**Arkivet, fordi papirkurven er det forkerte redskab.** Den siger "det her var en
fejl, og om tredive dage er det væk" — så alt, der er værd at beholde, kan ikke
komme i den. Et arkiv på tolv hundrede importerede noter er præcis det tilfælde,
hvor de fleste er værd at beholde og næsten ingen er værd at kigge på; uden et
sted at lægge dem er listen hele arkivet for evigt, og at finde ugens note
betyder at læse forbi et årti.

En kolonne, ikke en mappe. Noterne afviste et andet hierarki i 0018, og
arkivering er ikke arkivering i den forstand: det er én oplysning om, hvorvidt en
note er aktuel, og den skal kunne besvares ved siden af `deleted_at`, som er den
anden af samme slags. `updated_at` røres ikke — at rydde op er ikke at skrive.

**Flere ad gangen.** ⌘-klik lægger til, skift-klik tager intervallet i den
rækkefølge, listen faktisk står i — ikke notens egen, som er en anden, når der er
sorteret eller søgt. Trækker man en note, der er markeret, følger hele
markeringen med; trækker man en, der ikke er, er det den ene, for ellers ville et
træk i en tilfældig række flytte halvtreds noter, man havde glemt var markeret.

Ét kald frem for et pr. note: efter en import af tolv hundrede er det at vælge
halvtreds det almindelige tilfælde. Hver note kontrolleres stadig for sig — et sæt
er en bekvemmelighed for den, der kalder, og aldrig en vej til noget, de ikke
kunne have rørt én ad gangen.

En kant *og* en tone på det markerede: `--accent-sunken` er meget lys i lyst tema,
og tre markerede rækker var svære at skelne fra tre, der ikke var — en dårlig ting
at være i tvivl om lige inden man trykker Slet.

**Kopiér-knappen spiste næsten den kode, den skulle kopiere.** Den ligger inde i
`<pre>` for at kunne stå i hjørnet af blokken, og `colourCode` læste
`pre.textContent` og malede resultatet tilbage — så et klik på "Kopiér" skrev
"Kopieret" ind i koden, og næste gemning lagde det i filen. Nu ét sted, der siger
hvad en kodeblok indeholder, brugt af både farvelægningen og oversættelsen til
Markdown: var de to uenige, ville forskellen ende i noten. Prøven klikker,
læser udklipsholderen og læser filen bagefter.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-20 11:03:02 +02:00
Kristian WindandClaude Opus 5 8aad7c13b0 perf(notes): listen sendte hele noter for at vise halvfems tegn af dem
Release / Build and publish image (push) Failing after 9m55s
Chrome frøs på notesiden, og der var to grunde.

**Nyttelasten.** Listen viser en titel og omkring halvfems tegn. Den fik hele
noter for at gøre det: med tolv hundrede blev svaret 3,9 MB, hvoraf otteoghalvfems
procent var brødtekst, ingen ville læse på den skærm — hentet, tolket og holdt i
hukommelsen hver gang siden blev åbnet. Nu 763 KB.

Kroppen tømmes frem for at blive forkortet, og det er den vigtige halvdel. Et felt,
der hedder `body` og indeholder et *stykke* af kroppen, er en fælde: editoren ville
åbne det, pausen ville gemme det, og noten ville være skåret ned til sit eget
uddrag af ingenting andet end at være blevet kigget på. Tom er en form, klienten
kan se efter — og den gør det og henter hele noten, når den åbner en.

Det kostede en adskillelse mere: hvilken række der er *valgt*, og hvilken note der
er *indlæst*, er nu to ting. Editoren indlæser sin tekst, når notens id skifter, så
hvis den fik uddraget først og hele noten bagefter, ville id'et være det samme
begge gange og den anden aldrig blive vist. Man ville sidde med en tom note, der
lige havde haft en titel.

**Tegningen.** Alle tolv hundrede rækker fik layout og maling, også de tusind
ingen kan se: 2,1 sekunder til første tegning. `content-visibility: auto` lader
browseren springe det over og tage det, når man ruller derhen — 0,95 sekunder, og
uden en eneste linje JavaScript, ingen rullematematik og ingen liste, der kan komme
ud af trit med sig selv.

**Og søgefeltet.** Det var en pille med et forstørrelsesglas tegnet ind som et
baggrundsbillede med farven skrevet i hånden — #8b918d, uanset tema — hvilket
gjorde det til det ene sted på siden, der ikke fulgte resten. Nu samme form som
søgefeltet i toppen, et rigtigt ikon i `currentColor`, en ryd-knap og et antal
fundne.

Det havde heller ingen pause: hver tast sendte en søgning gennem tolv hundrede
noter, så "kaffe" var fem forespørgsler, hvoraf de fire var forældede inden de kom
tilbage — og svarene kunne lande i den forkerte rækkefølge. Nu én.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-20 10:36:35 +02:00
Kristian WindandClaude Opus 5 3a02bc8304 fix(notes): søgningen svarede med det forkerte først, og et træk gjorde ingenting
Release / Build and publish image (push) Failing after 9m55s
**Søgningen sorterede efter dato.** Usynligt med ti noter og ubrugeligt med tolv
hundrede: de blev alle importeret samme aften, så `updated_at` siger ingenting, og
loftet skar listen af ved halvtreds vilkårlige rækker. At søge "Tesla" svarede med
en note om nginx først og noten, der hedder "Tesla Model Y", bagefter — hvilket
læses som en søgning, der ikke virker, fordi det fra brugerens plads er præcis
dét.

Nu bm25, med titlen vægtet ti gange kroppen: et ord i titlen er, hvad noten
*handler om*, mens det samme ord én gang i en lang krop er en henvisning. `fold`
vejer som kroppen, ellers ville et dansk match slå et præcist. Sammenføjet frem
for `rowid IN (…)`, fordi rangeringen hører til matchet, og en underforespørgsel
smider den væk. Fladen sorterer ikke om, når der er søgt — det ville kaste netop
det væk, der gør en søgning i tolv hundrede noter brugbar.

**Et træk af en note hen på et projekt gjorde ingenting.** `ondragover` tog imod
både opgaver og noter, så rækken lyste op — og `ondrop` sendte kun opgaver videre,
mens noten faldt ned i grenen, der omarrangerer projekter, hvor den ikke betyder
noget. Den værst mulige form for i stykker: den så ud, som om den ville virke,
helt indtil man slap.

Det står i overleveringen som "virker i koden, men ikke fastholdt i prøver, fordi
prøven blev upålidelig". Den er skrevet igen, og den ser på resultatet — ligger
noten i projektet bagefter — frem for på om rækken blev fremhævet, hvilket var
netop det, den gamle kunne se og som blev ved med at være sandt, mens funktionen
var i stykker.

**Og datoen i listen var altid `updated_at`.** For tolv hundrede importerede noter
er det i aftes, så en note fra 2017 stod som "i går", og den dato, man sorterede
efter, var ingen steder at se. Rækken viser nu den dato, listen er sorteret efter,
og har begge i sit værktøjstip.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-20 09:58:41 +02:00
Kristian WindandClaude Opus 5 4aa8bd6a40 fix(sikkerhed): resten af gennemgangen — SSRF, tre ubegrænsede veje, og fire hemmeligheder
Release / Build and publish image (push) Failing after 9m54s
**H2 — SSRF gennem AI-udbyderens adresse.** `base_url` blev gemt uden kontrol og
hentet med en almindelig klient, og udbyderens svar blev givet tilbage ordret.
Det var en fungerende skanner af det interne net for enhver bruger: peg den på en
vært, læs de første tre hundrede bytes af det, der svarede. På en homelab er det
det meste af, hvad der er værd at nå.

Nu gennem `safedial`, og svarets krop går i loggen frem for i svaret. Operatøren
har stadig brug for beskeden — for en lokal model er den som regel selve
problemet — så den er der, bare kun for den, der kører instansen.

**M2 — Web Push havde tre.** Endepunktet blev gemt ukontrolleret og POSTet til på
en tidsplan, den, der ringer, selv vælger: sæt en påmindelse et minut ude, og
serveren henter adressen, hvert minut. Upserten flyttede desuden ejerskabet med
`user_id = excluded.user_id`, så enhver, der kendte en andens endepunkt, kunne
overtage det — offerets telefon ville få angriberens beskeder og ingen af sine
egne. Og `DeletePushSubscription` havde ingen ejer-kontrol.

**H3 — tre veje uden loft.** FTS-udtrykket byggede omkring halvtreds bytes pr.
ord uden grænse; en søgning på en kvart megabyte blev til to megabyte
MATCH-udtryk og seksogtyve sekunder inde i SQLite, mens den holdt på én af kun
fire til otte forbindelser. Nogle få af dem samtidig, og databasen er utilgængelig
for alle på instansen, logins inklusive. `filterquery.Parse` er kvadratisk —
halvtreds tusind led er syv sekunders CPU og seksogfyrre gigabyte allokering i én
forespørgsel — og havde hverken loft på antal eller på dybde. Og noteimporten
læste hver fil i arkivet uden grænse: 64 MB komprimeret kan blive til
seksogtres gigabyte skrevet ned på den disk, databasen ligger på.

**M1 — `comment_id` blev taget på tro.** Rettighedskontrollen så på opgaven i
adressen, mens filen blev hængt på den kommentar, formularen bad om. En viewer på
et delt projekt kan liste dets kommentarer, så hun kunne læse et id og lægge en
fil under en andens kommentar i et projekt, hun kun må læse.

**M4 — fire hemmeligheder lå i klartekst i hver sikkerhedskopi.** TOTP-koden og
AI-nøglen forsegles nu; begge konverterer sig selv næste gang de skrives.
Kalenderfeedets og mailadressens tokens gør ikke — de slås op på deres præcise
værdi og vises i fladen mere end én gang, hvilket en hash ikke kan. Det står nu
skrevet i `SECURITY.md`, som påstod det modsatte. At rette det betyder, at
adressen kun kan vises én gang, og det er en beslutning om, hvordan funktionen
virker.

Og en prøve på, at politikken overhovedet bliver sendt. Det var den, der gjorde
XSS-en i nat til indsat HTML frem for kørt script, og der var ingen prøve, der
rørte den — en foranstaltning, ingen prøve rører, kan forsvinde i en oprydning,
uden at nogen opdager det. Målt på script-src alene: `style-src 'unsafe-inline'`
står der med vilje, og en indsat stil kan gøre en side grim, ikke køre kode.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-20 08:39:52 +02:00
Kristian WindandClaude Opus 5 ce3fc466b2 fix(auth): andenfaktoren kunne gættes igennem, og et token kunne tilmelde sin egen
To huller fra sikkerhedsgennemgangen, og de er samme historie fortalt to gange:
den ene kontrol, der findes til når adgangskoden allerede er tabt, kunne omgås af
netop den, der havde adgangskoden.

**H1 — spanden blev fyldt af adgangskoden alene.** `handleLogin` nulstillede
ratebegrænseren ved hver korrekt adgangskode, *før* den så på, om der var en
andenfaktor. Det gav én, der havde kodeordet, ubegrænsede portioner på ti gæt:
log ind, få en frisk spand og en frisk ventende session, brug ti gæt på koden, log
ind igen. Med tre gyldige koder i ethvert øjeblik og tyve forespørgsler i
sekundet er det lige odds inden for en eftermiddag.

Spanden ryddes nu først, når login er *færdigt*, og der er kommet en spand mere:
koder tælles pr. konto og ikke kun pr. adresse. Adressespanden er slået af enhver
med mere end én adresse, hvilket er det almindelige — en telefon og en bærbar er
allerede to. Det, der gør en sekscifret kode sikker, er at gæt er en knap
ressource, og knapheden skal tælles imod det, der bliver angrebet.

Den ventende session får lov at leve. At afslutte den ved første forkerte ciffer
blev overvejet og valgt fra: det ville koste en, der taster ét tal forkert, sin
adgangskode igen, og det køber næsten ingenting oven i loftet pr. konto.

**M3 — et lækket token kunne tilmelde sin egen andenfaktor.** `/auth/totp/setup`
lå kun bag `requireAuth` og spurgte om ingenting, mens `disable` altid har
krævet adgangskoden: døren åbnede med et token og lukkede kun med et kodeord. Et
stjålet token kunne hente hemmeligheden, lægge den i sin egen godkender, bekræfte
den og beholde den eneste kopi af gendannelseskoderne. Ejerens næste login ville
stoppe ved en kode, de ikke havde, og de kunne ikke slå den fra uden at logge ind
først. Et tyveri bliver til et lejemål — den sætning står allerede i filen ved
siden af om passkeys, og det er den samme regel.

Nu bag `requireSession` sammen med passkeys og sessionerne, og `setup` spørger om
adgangskoden ligesom `disable`. Symmetrien er pointen: at slå den til er mindst
lige så indgribende som at slå den fra.

Og en note kan oprettes inde fra et projekt. At ligge i et projekt *er* måden en
note deles på, så det er én handling frem for to — man skulle før oprette noten
under Noter og bagefter finde projektet frem i "Del i". Ruden findes nu, også før
der er en eneste note, for ellers er der ingen knap dér, hvor man opdager at der
skal skrives noget ned.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-20 08:29:43 +02:00
Kristian WindandClaude Opus 5 bdde9cc6a5 merge: Google Kalender som menupunkt, med Verdandes egen kalender lagt over
Release / Build and publish image (push) Failing after 9m58s
Læs bredt, skriv snævert — decision 5 i planen, og kun den første halvdel:
kalenderen læses, og der skrives ingenting tilbage.

**OAuth-blokeringen er reel og blev ikke gået udenom.** En klient registreret som
*Internal* tager kun konti i sit eget workspace — også til kalendere, ikke kun
Gmail. Andreas' private konto bliver afvist med `org_internal`, og det er
klienten, der virker efter hensigten, ikke en fejl. Fladen siger det nu med ord,
man kan handle på, frem for at vise Googles egen skærm, der ikke gør.

Vejen udenom, som ikke er bygget: kalenderens egen hemmelige ICS-adresse. Ingen
OAuth, virker for enhver konto, og `internal/ics` findes allerede. Det er det
næste stykke, og det eneste, der låser den private konto op.

`internal/google` er flyttet ud af `internal/gmail`, fordi de to er det samme
flow, der er uenige om én streng. Egen callback-adresse frem for at dele Gmails:
én callback, der finder ud af hvilken token-butik den skal skrive i, går galt ved
at overskrive en Gmail-forbindelse, der virkede.

Egne tabeller frem for `mailboxes`. `trigger_kind`, `seen` og `last_uid` svarer
alle på "hvad gør en besked til en opgave", og ingenting her bliver til en
opgave; én konto har mange kalendere, og `mailboxes` har ikke plads til en
en-til-mange. Afgørende i praksis: `mailboxes.kind` har en CHECK, og SQLite kan
ikke ændre en — en tredje værdi koster den tabelombygning, som migration 0021
skulle igennem.

Begivenheder erstattes pr. kalender pr. vindue frem for at flettes med et
sync-token: et vindue, der slettes og skrives forfra, kan ikke komme ud af trit,
og et sync-token er endnu en ting, der skal være rigtig for altid, for at gitteret
ikke stille viser noget forkert.

`CalendarView` er udvidet, ikke kopieret. En begivenhed er et fyldt bånd, hvor en
opgave er et kort med en farvet kant, så de to læses forskelligt før et ord er
læst — og den kan ikke trækkes og er aldrig en knap: der er ikke noget at gøre
færdigt ved en andens møde.

To værn på det, der kommer fra Google: farven skal matche et hex-mønster, før den
må stå i en style, og linket bliver kun til et href, hvis det begynder med https.

Konflikten i jobs.go var to nye jobs skrevet på samme linje. Begge skal med.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-20 07:39:12 +02:00
Kristian WindandClaude Opus 5 6b2cd0f259 merge: engelske skærmbilleder, og otte steder hvor fladen blev ved med at være dansk
Release / Build and publish image (push) Failing after 9m54s
Skærmbillederne er engelske, fjorten i alt, og der er to mennesker i dem. Det
sidste er ikke pynt: halvdelen af det, programmet er til, findes ikke med én
konto — en ansvarlig, et navn i historikken, en medlemsliste, og "Venter på
andre" som andet end en tom side.

Sproget kommer fra kontoens locale og ikke fra strenge skrevet i hånden, hvilket
er dét, der fandt fejlene: otte steder, hvor grænsefladen blev ved med at være
dansk, uanset hvad kontoen sagde. "Forsinket" og "I dag" skrevet ind i markup,
"Uden sektion" som boardets første kolonne, gentagelsesreglerne halvt oversat,
alle nitten navne på hændelser i historikken, og et `da-DK` hårdkodet i Kommende.
Gentagelsesreglerne er flyttet til `when.js`, så menuen og mærket ikke kan blive
uenige igen — det er præcis, hvad den fil er til.

README's billeder havde været brudte siden `37605ba`, som lavede PNG'erne om til
WebP og slettede dem, uden at teksten fulgte med: forsidebilledet og alle seks i
tabellen var 404 på GitHub. Og noterne — hele funktionen — stod ikke nævnt hverken
i README, på forsiden eller i dokumentationen.

Nu skrives PNG til docs og WebP til site, altså præcis dét, de to steder læser.
Før efterlod hver kørsel tolv usporede PNG'er i site, som intet pegede på.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-20 07:34:16 +02:00
Kristian Wind dcd66b415e docs(web): registreringsteksten fortæller nu, at klienten dækker begge dele
Hjælpeteksten sagde "slå Gmail API til", og det er kun det halve nu. Den samme
klient bærer både Gmail og Google Kalender og skal have begge redirect-URI'er —
og kalenderpanelet siger, hvor dets egen kommer fra, så ingen leder efter et
felt til en klienthemmelighed, der allerede står ovenfor.
2026-08-20 07:33:24 +02:00
Kristian WindandClaude Opus 5 3991ec9445 feat(notes): et billede kan indsættes og trækkes ind i en note
Rapporteret fra brug: man kunne ikke indsætte billeder eller trække filer ind i
noter. Det passede — importen kunne tage billeder med, og intet andet kunne.
Indsætningen tog kun `text/plain` og smed resten væk, og der var slet ingen
håndtering af et træk. At indsætte et skærmbillede er det mest almindelige, der
findes i et noteprogram, og det gjorde ingenting overhovedet.

Der manglede også et sted at lægge filen. Vedhæftninger på en note fandtes i
databasen, siden migration 0021, men kun importen kunne skrive dem; der var ingen
rute. Nu er der en, med editor-rettigheder på noten og samme svar for "findes
ikke" og "ikke din", så et id ikke kan afprøves.

Skrevet ind som `![](…)` og ikke som en data-URL: en note er Markdown i en
database, og et fotografi fra en telefon som base64 midt i teksten ville gøre
noten til fire megabyte tekst, der skal læses hver gang listen tegnes.

`execCommand` frem for at bygge noden selv, fordi det er dét, der lægger
handlingen i browserens egen fortryd-historik — ⌘Z tager billedet ud igen,
ligesom det tager et ord ud.

`preventDefault` på både dragover og drop. Uden den første kommer der intet drop;
uden den anden åbner browseren filen i stedet, og man mister det, man var i gang
med at skrive, til en fane med et billede i. Kun filer: et træk inde fra appen
selv bærer vores egen nyttelast og skal falde igennem til den, der lyttede.

Prøven går begge veje ind, ser at billederne bliver hentet og tegnet, og læser
filen bagefter for at sikre, at der står to bilagsadresser og ingen data-URL.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-20 07:32:03 +02:00
Kristian Wind a5f2f769c4 fix(web): begivenhederne i en celle sorteres, hvor de tegnes
Serveren sorterer allerede, men en celle, der viser 14:00 over 09:15, ligner
ikke en liste i forkert rækkefølge — den ligner et forkert klokkeslæt på
chippen, og det er langt værre at skulle udelukke. Heldags først: en heldags
begivenhed er et bånd hen over dagen, ikke noget der sker på et punkt i den.
2026-08-20 07:30:55 +02:00
Kristian WindandClaude Opus 5 9d743a5d46 fix(tools): shots skrev PNG'er ned i mappen med WebP
Standardstien var site/screenshots, som siden landingssiden gik over til WebP
kun indeholder WebP. En kørsel lagde tolv PNG'er ved siden af dem, som ingen
tekst peger på. Den skriver nu til docs/, hvor de andre PNG'er ligger.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-20 07:27:48 +02:00
Kristian Wind 031bb18067 fix(calendar): en tom kalender skal sige hvorfor, ikke bare være tom
At vælge en kalender er nogen, der siger "vis mig den her". At gemme valget og
lade gitteret stå tomt, til fejeren kommer forbi et kvarter senere, er en
kalender, der påstår, at fjorten dage er fri — og personen står på siden med
knappen under hånden. Gemningen henter nu selv.

Og er der forbundet en konto uden at der er valgt nogen kalendere, siger
visningen dét. "Der er ikke valgt en kalender" og "der er ikke booket noget"
ser ens ud, og kun den ene er noget at gøre noget ved.
2026-08-20 07:27:42 +02:00
Kristian WindandClaude Opus 5 71a9fe63d6 docs: noterne findes også i teksten, og hvert billede har et sted at høre til
README'ets billeder havde været brudte siden landingssiden gik over til WebP:
site/screenshots/*.png blev slettet dengang, og markup'en pegede stadig på
dem. Den peger nu på PNG'erne i docs/, som er dem, der findes.

Noter er en af de største ting i appen og stod ikke ét sted i teksten —
hverken i README'et, på landingssiden eller i dokumentationen. Nu gør de,
med de to nye billeder.

Fem billeder pegede ingen tekst på: upcoming, sections, delegated, settings
og admin. De har fået et afsnit i docs/index.md, hvor hvert enkelt beviser
noget, siden siger. Landingssidens gitter viste den lyse udgave to gange —
den ene af dem er blevet til noterne.

Og to billedtekster passede ikke længere på billedet: temaerne viser nu også
de fire udseender, og opgaven i skuffen viser ikke sin samtale.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-20 07:25:43 +02:00
Kristian WindandClaude Opus 5 eeaaf7506b docs: skærmbillederne er engelske, og der er to mennesker i dem
Landingssiden og dokumentationen læses på engelsk, og billederne var danske
hele vejen — både grænsefladen og indholdet. Kontoen bliver nu oprettet med
engelsk sprog frem for at oversætte noget her: det er sådan appen selv gør
det, og det er også dét, der får en glemt oversættelse til at stå i et
billede i stedet for at vente på en bruger.

Demodataene er skrevet om til engelsk arbejde og husholdning. Og der er en
person mere: en invitation, en konto, tre opgaver hun sidder på og en
kommentar. Halvdelen af det, appen er til for, findes først, når der er to i
den — en ansvarlig på en opgave, et navn i historikken, og "Venter på andre"
som andet end en tom side.

Tre billeder viste ikke det, teksten sagde. `detail` var en tom formular, så
opgaven har fået beskrivelse, undertasks, en fil, en påmindelse og en samtale.
`themes` viste profilfeltet over temaerne, så den ruller nu ned til dem.
`delegated` var en tom tilstand.

Noterne er kommet til — listen med en note åben, og den samme note som kilde.
Fjorten billeder i alt.

WebP'en skrives af den browser, der i forvejen står åben, frem for af cwebp:
et program mere at installere hører ikke til i et script, hvis hele pointe er,
at der ikke er noget at installere. PNG'erne ligger nu kun i docs/ og WebP'en
kun i site/, hvilket er præcis dét, de to steder læser — før lagde et kørsel
tolv usporede PNG'er i site/, som ingen brugte.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-20 07:24:12 +02:00
Kristian Wind 1b1e57df29 docs: kalenderen den anden vej, og én registrering til begge Google-ting
`caldav.md` beskrev to måder at sende opgaver *ud*. Den tredje er den modsatte
retning og hører til samme sted, fordi det er dér, man ville lede.

`configuration.md` hed "Gmail" og handler nu om Google: samme klient, to
redirect-URI'er, to API'er, to scopes. Bemærkningen om en Internal-app står
begge steder, fordi den gælder begge veje — og for én privat konto er
kalenderens egen hemmelige ICS-adresse den kortere vej. Intet i verdande
abonnerer på sådan en endnu.
2026-08-20 07:24:01 +02:00
Kristian WindandClaude Opus 5 dce8800f93 fix(web): otte steder hvor grænsefladen blev ved med at være dansk
Skærmbillederne skulle tages på engelsk, og det var dét, der fandt dem:
en konto sat til engelsk fik stadig "Forsinket" over de forsinkede opgaver,
"Uden sektion" over den første kolonne på tavlen, "Aldrig" i gentagelses-
menuen, "hver dag" på rækken, "til instansen · inviteret af" på invitationen,
"3 projekter" på kontoen, og hele historikken — nitten hændelser, alle på
dansk. En funktion, der kun virker på det ene sprog, er en funktion, der er i
stykker.

Gentagelsen er den eneste, der ikke bare manglede en nøgle. Serveren skriver
`recurrence_text` på dansk og skal blive ved med det: én besked går over
WebSocket til alle i projektet på én gang, og de læser ikke det samme sprog.
Så de regler, der har et navn, får det i when.js — på den side, der ved, hvis
skærm det er — og alt mere specifikt falder tilbage til serverens ord.

Mellemrummet i historikken er skrevet ud. Svelte fjerner det, der åbner en
blok, så "inviterede" og adressen løb sammen til ét ord.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-20 07:23:42 +02:00
Kristian Wind c3f2181efe feat(web): Kalender som menupunkt, med Googles begivenheder lagt over egne opgaver
Samme gitter som Kommende, givet en liste mere at tegne. Ikke et gitter nummer
to, og grunden er den del, der er nem at få næsten rigtig: et mål skal afgøre
ud fra `dataTransfer.types` alene, om det tager imod, og to kopier af det er to
steder at være næsten rigtig. En opgave trukket her flyttes præcis som alle
andre steder, fordi det er den samme kode, der gør det.

En begivenhed er ikke en opgave, og cellen siger det før et eneste ord er
læst: en flade af farve, hvor en opgave er et kort med en farvet kant. Den kan
ikke trækkes — forbindelsen er læsende, og en chip, der letter fra siden og så
ikke vil lande, læses som en fejl frem for som en regel — og den er ikke en
knap, for der er ikke noget at lukke ved en andens møde. Farven er Googles egen
for den kalender, så to kalendere i ét gitter kendes fra hinanden på den farve,
personen allerede kender dem på.

Farven tjekkes frem for at blive troet: den ender i en inline `style`, og en
værdi, der ikke er en farve, kan lukke erklæringen og åbne en anden. Linket
tegnes kun, hvis det begynder med `https://` — et href er dér, hvor en
`javascript:`-adresse bliver til et klik, der kører.

Prøven bruger uge-visningen med vilje: en månedscelle skæres af ved tre chips,
og resten af suiten lægger opgaver på andre dage. Datoen læses ud af siden, ikke
ud af testprocessen, som de øvrige har lært at gøre.
2026-08-20 07:22:25 +02:00
Kristian WindandClaude Opus 5 241c5ed37d fix(mail): den samme mail blev en opgave igen og igen
Release / Build and publish image (push) Failing after 9m58s
Rapporteret fra drift med et skærmbillede af den samme mail seks gange i træk.

**IMAP: flaget kommer af, når mailen er blevet en opgave.** Markøren var et uid —
husk det højeste, begynd over det næste gang — og den holdt kun, hvis uids var,
hvad serveren sagde. Den brød på et fetch-svar uden uid, på en UIDVALIDITY, der
skiftede under en genopbygget postkasse, og på to postkasse-rækker, der pegede
samme sted og hver især førte deres eget regnskab. Alle tre gav det samme
symptom.

Så regnskabet flytter derhen, hvor spørgsmålet bliver stillet. "Flaget" betyder
"lav det her om til en opgave"; når den er en opgave, har flaget gjort sit
arbejde og er ikke sandt længere. Så er der ingenting at huske, og dermed
ingenting, der kan komme ud af trit — en anden synkronisering, en anden
postkasse-række eller en database gendannet fra en kopi finder alle det samme
svar, fordi svaret ligger i postkassen.

Flaget tages af *efter* opgaven er skrevet. Et flag fjernet fra en mail, hvis
opgave aldrig blev gemt, er en mail, ingen ser igen.

**Gmail havde den samme fejl af en anden grund.** Listen over allerede behandlede
beskeder blev bygget om ved hver kørsel af *kun de id'er, Gmail svarede med den
gang* — for at den ikke skulle vokse uendeligt. Men forespørgslen er begrænset,
så på en konto med mere stjernemarkeret post, end der er plads til i ét svar,
glemte hver kørsel det, den forrige havde husket. De kom tilbage som ubehandlede
og blev opgaver igen. Det var ikke en grænse, det var en måde at glemme på.
Listen har nu et loft på antal i stedet, nyeste først.

Gmail kan ikke få taget stjernen af: scopet er `gmail.readonly`, med vilje, fordi
det er dét, der gør samtykkeskærmen til noget, et menneske kan sige ja til. Derfor
skal regnskabet blive på vores side dér — men det skal så være et regnskab, der
kan holde.

Prøven kører hele vejen mod en rigtig IMAP-server: find de flagede, tag flaget af,
og spørg så forfra med markøren sat til nul — altså det værst tænkelige tilfælde,
hvor alt, vi selv havde husket, er væk. Der kommer ingenting tilbage.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-20 07:04:40 +02:00
Kristian Wind 987d4edfce feat(calendar): en læst Google-kalender, med sit eget endpoint og sin egen sweep
Samme registrering som Gmail, et andet scope og en anden redirect-URI. Google
udsteder en refresh-token pr. godkendelse og holder op til hundrede i live pr.
konto, hver med sine egne scopes — så en forbundet kalender rører ikke en
Gmail, der virker, og at afbryde den ene tager ikke den anden med.

Callbacken har sin egen sti. Én callback, der ud fra gemt tilstand afgjorde,
hvilket af de to flows der kom tilbage, ville lægge valget af *hvilket
token-lager der skrives* bag et opslag — og går det galt, går det galt ved at
overskrive en virkende Gmail-forbindelse med kalender-tokens.

Begivenhederne caches i et vindue på 90 dage tilbage og et år frem. Et
månedsgitter bladrer, og et kald til Google pr. piletast er en kalender,
der er langsommere end den kalender, den viser. Vinduet har en kant, og
endpointet siger hvor den er: et tomt svar uden forklaring er en kalender,
der lydløst påstår, at en dag er fri.
2026-08-20 07:01:14 +02:00
Kristian WindandClaude Opus 5 78891bae6f feat(beacon): tæl installationer, sig præcis hvad der sendes, og lad folk sige nej
Spørgsmålet er ét: hvor mange bruger det her. Svaret er to værdier én gang i
døgnet — et tilfældigt id, installationen selv fandt på, og versionen. Ikke andet.
Beskeden er lille nok til at stå i fuld længde på indstillingssiden, og den gør
det, fordi et løfte om telemetri er præcis så meget værd som læserens mulighed
for at efterprøve det.

**Ikke obligatorisk, og grunden er praktisk frem for principiel:** det kan ikke
håndhæves. Domænet kan blokeres, binæren kan patches, og en fork fjerner den i
første commit — så obligatorisk køber intet over slået-til-som-udgangspunkt og
koster den tillid, resten af programmet arbejder for. Hemmeligheder forsegles
her, nøglen holdes uden for databasen, og sikkerhedskopisiden advarer om, hvad en
kopi indeholder. Telemetri, man ikke må sige nej til, er den ene sætning i den
historie, der ikke passer.

Uenigheden med `config.UpdateCheck`, som er slået fra med den modsatte
begrundelse, står skrevet i pakkens egen dokumentation frem for at blive glattet
ud. Den er reel, og læseren fortjener at se begge halvdele.

Id'et findes på og gemmes ved første brug frem for at blive udledt af værtsnavn
eller adresse. En hash af noget om maskinen ville være stabil, bekvem og
samtidig en værdi, modtageren kunne holde op mod et gæt — og det er forskellen
mellem anonym og pseudonym.

Modtagerens side er den samme binary med en kontakt: `POST /api/v1/beacon` uden
login, fordi den, der ringer, er en fremmed installation. Den svarer 404, når
instansen ikke er tælleren, så en almindelig installation ikke er en spamspand,
der svarer 200 på alt. Ratebegrænset som login-ruterne, og tabellen har et loft:
endepunktet indsætter en række på et id, den, der ringer, har fundet på, og uden
et loft kunne enhver fylde disken.

Tallet er "set inden for tredive dage" og ikke et løbende total. Et total stiger
kun og holder op med at betyde "hvor mange bruger det" og begynder at betyde
"hvor mange har prøvet det".

`internal/safedial` kom til undervejs: modtagerens adresse kan skrives i fladen,
og en adresse, en bruger angiver, må ikke kunne pege indad. Kontrollen ligger på
den opslåede adresse og ikke på URL'en, fordi et navn kan svare 127.0.0.1 — og
svare noget andet anden gang, efter en kontrol ved parsning er gået godt.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-20 06:59:50 +02:00
Kristian Wind 7828c1e6ed feat(gcal): Google Calendar kan læses, og kalenderen får sine egne tabeller
Klienten henter kalenderlisten og begivenhederne i et vindue, og laver Googles
to former om til den ene, et gitter kan spørge om: dage i begge ender, begge
med. Googles heldags-slutdato er eksklusiv, og et møde, der slutter ved midnat,
slutter aftenen før — begge dele rettes ét sted frem for hos hver læser.

Egne tabeller frem for `mailboxes`. De to ligner hinanden på afstand og er
uenige om hver eneste kolonne bagefter: `trigger_kind`, `seen`, `last_uid` og
`project_id` svarer alle på "hvad laver en besked om til en opgave", og intet
her bliver til en opgave. Én konto rummer desuden mange kalendere, og en
en-til-mange er der ikke plads til. Det afgørende er mindre og mere praktisk:
`mailboxes.kind` har en CHECK, og SQLite kan ikke ændre en CHECK — en tredje
værdi koster en tabelombygning med fremmednøgler slået fra. Derfor er der ingen
CHECK på `provider` her: CalDAV skal koste en række, ikke en ombygning.
2026-08-20 06:50:57 +02:00
Kristian Wind a46d9158e1 refactor(google): OAuth-flowet flytter ud af gmail, så to ting kan dele det
Gmail og Google Calendar logger ind gennem den samme registrering med den
samme PKCE og den samme fornyelse. Det eneste, der adskiller dem, er ét ord:
scopet. En kopi mere ville være et sted mere, hvor `access_type=offline` kan
blive glemt — og det er dén fejl, der får en forbindelse til at dø lydløst en
time efter, den blev lavet.

internal/gmail beholder sit API og sit scope; navnene på flowet er aliaser, så
hverken kaldere eller prøver ændrer sig. Prøverne for PKCE, AuthURL og
Configured flytter med koden.
2026-08-20 06:46:10 +02:00
Kristian WindandClaude Opus 5 4b5f3f0864 chore: agenternes arbejdstræer hører ikke i repoet
Release / Build and publish image (push) Failing after 9m56s
Et `git add -A` fangede .claude/worktrees/ som et indlejret repo. Det er en
agents arbejdskopi og skal ingen steder.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-20 06:39:45 +02:00
Kristian WindandClaude Opus 5 052f8a43fb fix(notes): en note kunne smugle et script ind gennem et billedes alt-tekst
Indført i aften, sammen med billedvisningen, og fundet ved at læse den igen.

`escapeHtml` tager &, < og > — ikke anførselstegn, og det er rigtigt for tekst
mellem to tags, hvor et anførselstegn ikke betyder noget. `alt` er ikke tekst
mellem to tags. Det står inde i en attribut, og der lukker det første
anførselstegn den, hvorefter resten står som markup:

    !["  onerror="alert(1)](/api/v1/attachments/aaaa)

blev til et <img> med en onerror på. Adressen var i forvejen bundet til vores
eget bilagsmønster, så den var ikke vejen ind — teksten ved siden af var.

Det er ikke en teoretisk vej. En note deles ved at ligge i et projekt, så den
læses af alle, der kan læse projektet, og scriptet ville køre på vores eget
domæne med sessionscookien, hos den der åbnede noten.

En værdi i en attribut har brug for sin egen undslipning, og har nu en. Prøven
skriver noten, åbner den og ser efter, at hændelsen står som tekst i alt og ikke
som en attribut på elementet — og at billedets 404, som er selve vejen en
onerror ville blive kaldt ad, ikke kalder noget.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-20 06:39:19 +02:00
Kristian WindandClaude Opus 5 dee2d5e30b fix(faerdige): en opgave lukket efter midnat lå under "I går"
Release / Build and publish image (push) Failing after 9m56s
Fundet af prøverne, som for første gang kørte forbi midnat.

Dagen blev læst som `completed_at.slice(0, 10)` — de ti første tegn af et
tidsstempel, der er i UTC. Overskriften sammenlignede med *lokal* dato. I dansk
sommertid er de to ikke den samme dag i de første to timer efter midnat, så en
opgave lukket 00.30 blev sat under "I går", indtil klokken blev to. Tyve timer i
døgnet er de ens, hvilket er derfor prøven har været grøn hver eneste gang før.
Dagen udregnes nu lokalt, som den overskrift den skal svare til.

Og prøven for invitationslinket kunne kun bestå seks dage om ugen. Den ledte
efter "males inden fredag" som én tekst, men hurtig tilføjelse læser "fredag" som
forfaldsdatoen og tager ordet ud af titlen — tilbage står "males inden" plus et
mærke, hvis ord skifter med ugedagen. Kørt på en torsdag hedder mærket "I
morgen", og så fandt den ingenting. Prøven handler om invitationer, ikke om
datoer, så den ser nu på titlen alene.

Uddraget i notelisten viste `![](/api/v1/attachments/01a0…` for hver note med et
billede i — den eneste tekst i listen, ingen havde skrevet.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-20 00:17:50 +02:00
Kristian WindandClaude Opus 5 2bd8e48aba fix(notes): en note med tre prikker i titlen blev smidt væk ved importen
Release / Build and publish image (push) Failing after 9m54s
`skipFile` afviste ethvert navn, der indeholdt "..", for at stoppe en zip i at
skrive uden for sin egen mappe. Det er den rigtige idé rettet mod det forkerte:
det beskriver ikke en sti, der klatrer ud af arkivet, men ethvert filnavn med to
prikker ved siden af hinanden. En ellipse i en overskrift er almindelig, og noten
"Så blev det endelig jul! Stay tuned... ☕️" blev derfor aldrig oprettet.

Lydløst, oven i købet: filen blev sprunget over, før den blev talt, så svaret
sagde det samme, som hvis den var kommet med. Én note ud af tolv hundrede, og den
blev kun fundet ved at tælle efter, hvor mange der kom ud i den anden ende — 1140
hvor der skulle stå 1141.

Nu læses ".." som et stiled, ikke som to tegn, og en Markdown-fil, filteret
afviser, tælles som sprunget over. En import, der taber noget, skal sige det.

Og move-ikonet i sidebjælken er væk. ⠿ sad i den to pixels brede rende til
venstre for teksten og lå oven i rækkens egen kant — idéen var rigtig, pladsen
var det ikke. Elementet bliver: det er dét, der ejer trækket, og gjorde man i
stedet hele rækken draggable, ville alt indeni også blive det, og en opgave
trukket hen på I dag holdt op med at lande. Nu er håndtaget usynligt og lidt
bredere, og hånden ved hover er hele forklaringen.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-20 00:04:08 +02:00
Kristian WindandClaude Opus 5 9b4e7686f9 feat(notes): datoerne følger med, billederne bliver tegnet, og listen kan sorteres
Release / Build and publish image (push) Failing after 9m54s
Fire ting, som en rigtig import af tolv hundrede noter fandt på en time.

**Billederne blev vist som ord.** `markdownToHtml` havde ingen regel for
`![](…)` overhovedet, så en note fra Apple Noter viste sin egen billedadresse som
tekst. Filen var hentet, gemt og hæftet på noten; det eneste, der manglede, var
at nogen tegnede den. Kun vores egen adresse bliver til et billede — et
`![](http://…)` fra en indsat tekst ville ellers hente fra en fremmed vært, når
noten åbnes, og en note er det sidste sted, man vil have en sporingspixel. Og
vejen tilbage skulle med: uden `IMG` i inlineToMarkdown ville den første gemning
have taget hvert billede ud af noten igen.

**Titlen blev revet i stykker.** Noter deler en linje op i så mange stumper, den
har brug for — "Aalborg Kommune" står som
`<h1>A</h1><h1>alb</h1><h1>org Kom</h1><h1>mune</h1>`, fordi der er en
attributkørsel bag hver stump — og eksporten læste et linjeskift ved hver af dem.
Noten kom ud som fire overskrifter oven på hinanden, altså ødelagt på den mest
synlige måde der findes: i sin egen titel. Samme rod gav `**M****øde**` for fed
og `*M**øde*` for kursiv, hvor de to stjerner i midten læses som fed. Nu brydes
der kun, når blokken skifter, og et mærke, der lukkes og straks åbnes igen, er
ingenting.

**Datoerne fandtes ikke.** Eksporten læste dem aldrig, og Markdown havde ingen
plads til dem — så en flytning af tolv hundrede noter blev en bunke, der alle var
skrevet samme aften. Rækkefølgen er halvdelen af, hvad et arkiv er. Nu står de i
et YAML-hoved, som Obsidian og Bear allerede skriver, og importen læser det.
`SetNoteTimes` frem for gennem SaveNote: `updated_at` betyder "hvornår skrev
*dette* program noten sidst", og lod man enhver kalder sætte den, ville kolonnen
betyde det, den sidste skriver havde lyst til.

**Og listen kan sorteres** — senest rørt, oprettet, navn. Favoritter står øverst
uanset hvad; det er hele meningen med at gøre en note til favorit.

Eksporten henter nu i klumper. Ét kald pr. note tog halvanden time for tolv
hundrede, ikke fordi Noter er langsom, men fordi hvert kald er en proces, der
skal startes — og det gjorde det umuligt at rette en fejl i oversættelsen og se
resultatet, for hver runde var en frokostpause. Antallet af bilag kom med i
klumpen af samme grund: ni ud af ti kald var en proces startet for at få svaret
nul.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 22:58:19 +02:00
Kristian WindandClaude Opus 5 2ffbd9533b fix(notes): billederne i en importeret note pegede på ingenting
Release / Build and publish image (push) Failing after 9m54s
Fundet ved at importere rigtige noter frem for ti prøvenoter.

Linket blev skrevet halvt om. attachToNote kendte to måder, en note kan nævne en
fil på — den bare filnavn og stien i forhold til noten — men ikke den, en Apple
Noter-eksport faktisk skriver: `vedhaeftninger/751-2-IMG_0615.heic`, med noten i
roden ved siden af mappen. Kun det bare navn matchede, så kun det blev erstattet,
og hver eneste billede i hver eneste importerede note endte som
`vedhaeftninger//api/v1/attachments/…`, som er ingenting. Filerne var gemt og
hæftet rigtigt på; det var linket, der var forkert. Længste match først, ellers
bliver mappen stående foran adressen.

Og billederne kunne alligevel ikke vises. Alt blev serveret som
application/octet-stream med Content-Disposition: attachment, hvilket er en god
regel med en god grund — en uploadet SVG eller HTML vist inline ville køre sit
eget script på dette domæne med sessionscookien på. Men en note bærer sine
billeder i teksten som `![](…)`, og under den regel er hvert eneste af dem et
brudt billede. Hele importen findes for at tage dem med.

En kort liste af rasterformater vises nu inline med deres egen type. Det er ikke
en opblødning: listen er navngivet ét format ad gangen frem for matchet på
"image/", fordi præfikset ville lukke image/svg+xml ind — et dokument, der kan
køre script — og en liste, der vokser ved et uheld, er præcis sådan det sker.
nosniff står, så browseren skal læse filen som netop det billede eller slet ikke.

Titlen stod to gange i fjorten noter. Noter kalder noten det, der står på de
første linjer sat sammen med mellemrum, så en note, hvis to første linjer er
"CPU" og "CAN", hedder "CPU CAN" — og sammenligningen med kun den første linje
gik fri. Overskrifterne øverst tages nu med, så længe de tilsammen stadig er
begyndelsen på titlen.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 22:02:20 +02:00
Kristian WindandClaude Opus 5 f6c6bdccbb fix(notes): projektmenuen står ved det # der bliver skrevet
Den var lagt fast øverst i ruden, med den begrundelse at en menu, der følger
markøren, rykker sig et par pixels for hvert bogstav. Det er rigtigt, og det var
alligevel forkert: i en note på en skærmfuld skriver man `#` nederst og får en
menu i toppen — fire hundrede pixels fra det, øjet ser på, og langt nok væk til
at man ikke opdager, at den er der.

Begge dele kan lade sig gøre. Positionen måles, når menuen åbner, og ikke igen
mens listen snævres ind: den står ved tegnet, og den står stille.

Målt på `#term` frem for på markøren, fordi et sammenfaldet område ikke har nogen
kasse på en tom linje. Klemt inden for ruden, og vendt op over linjen, når der
ikke er plads under — ellers ville en note, der er skrevet helt ned i bunden,
lægge menuen uden for skærmen.

Og pak-noter-i-portioner.py, fordi en Apple Noter-eksport er hundrede og
enoghalvfems megabyte, hvoraf teksten er fire: importen tager 64 MB ad gangen, og
tunnelen foran den tager mindre. Delt efter hvad en note vejer med sine billeder,
og altid med billederne i samme portion som den note, der peger på dem.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 21:44:03 +02:00
Kristian WindandClaude Opus 5 9e687185fb feat(notes): vis noten som kode, og giv teksten en margen at ånde i
Release / Build and publish image (push) Failing after 9m59s
Kildevisningen, fordi den rige flade ikke kan tage imod Markdown. Indsat i den
er "## Overskrift" fem tegn og ikke en overskrift, og der var ingen vej ind for
tekst skrevet et andet sted — hvilket er en mærkelig mangel i et program, hvis
hele lager er Markdown. Knappen </> viser filen, som den ligger i databasen, og
det, man skriver der, går tilbage gennem den samme oversætter som altid.

Fyldt fra kilden og ikke fra note.body, når man går tilbage: gemningen venter på
sin pause, så den vej ville kaste det nyeste væk. Og kildevisningen hører til den
note, den blev slået til på — skifter man note, er det den rige flade, man møder.

Margenerne. Teksten lå helt ude ved kanten, og på en bred skærm blev linjerne
hundrede og fyrre tegn — dér mister øjet hvilken linje det var på, når det
springer tilbage. Arket har nu marginer, der vokser med ruden, og teksten en
grænse på 46rem. Kun teksten: værktøjslinjen og foden hører til ruden.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 21:28:16 +02:00
Kristian WindandClaude Opus 5 8229800767 fix(notes): lister overlevede ikke en gemning, og tallene blev klippet af
Seks ting fundet ved at bruge appen, i den rækkefølge de gjorde ondt.

En liste inde i en blok blev læst som tekst. htmlToMarkdown havde én gren for
ul og ol og lod alt andet falde ned i inlineToMarkdown, som samler efterkommere
til én streng — så et blokcitat med en liste i kom tilbage som "> PunktPunkt 2",
og en note fra Apple Noter, der er fuld af citater, mistede hvert punktmærke ved
den første gemning. Ingen havde rørt de linjer. Nu læses en blok, der rummer
blokke, igennem frem for flad, og et citat sætter "> " foran det, der kommer
indefra.

Indlejrede lister fandtes ikke. Den flade `list`-variabel kunne kun beskrive ét
niveau, så et indrykket punkt blev lagt ved siden af sit ophav. Nu en stak, og
underlisten lægges inde i sit <li> — det er den forskel, der kan læses tilbage.

En liste, der begyndte ved ti, begyndte ved ét bagefter. `start` skrives nu med
og læses med, så en gemning uden ændringer ikke gør noten om.

Tallet blev klippet af. Mærket sættes uden for tekstkanten, og 16px rakte til et
punkttegn og ikke til "10." — resten hang ud over kanten, hvor .page beskærer.
Renden er nu 2.2em og deles af begge slags, så punkter og tal flugter.

# viste seks vilkårlige projekter ud af tyve og så færdig ud. Loftet er væk,
listen kan rulle, og det, der begynder med det tastede, står øverst.

Og to ting om, hvordan det ser ud. Noteruden har fået sin egen flade, fordi et
ark, der holder op et sted, skiller det, man skriver, fra det program, man
skriver det i — `--surface` frem for `#fff`, så de fem temaer hver får deres.
Noter stod for sig selv over en streg, hvilket sagde det rigtige og kostede
halvfjerds pixels; den står nu i rækken med de faste visninger og bærer et ark i
stedet for en prik, hvilket siger det samme gratis.

Prøven skriver de fem listeformer, gemmer gennem den rigtige editor og læser
filen bagefter.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 21:24:43 +02:00
Kristian WindandClaude Opus 5 a4b12f3601 feat(notes): listen som Apple Noters, ⌘K finder noter, og de kan trækkes
Release / Build and publish image (push) Failing after 9m52s
Rækken har nu titel i samme størrelse som en opgavetitel — en note og en opgave er
to ting af samme slags, og hver sin størrelse får den ene til at se mindre vigtig
ud. Under den står dato og begyndelse på samme linje, som Apple Noter gør det: to
linjer pr. note frem for tre, og datoen først, fordi det er den, man skanner efter.
Ligger noten i et projekt, står det nedenunder.

⌘K søgte kun i opgaver og projekter. Feltet ser ud til at kunne finde alt, så det
er svært at gætte, at halvdelen af programmet ikke var med. Og mens jeg var der:
"Projekt" og "Opgave" stod hårdkodet på dansk i paletten.

En note kan trækkes ind på et projekt i sidebjælken — samme mål som en opgave,
fordi det er samme gestus: læg denne her.

To fejl i søgefeltet, begge Safaris egen type="search":
Fokusringen tvang sin egen hjørneradius og tegnede en firkantet bue uden om en
rund kant — to streger, hvor der skulle være én. Den følger nu elementets form,
overalt i programmet. Og feltet skiftede udseende ved fokus: forstørrelsesglasset
forsvandt og pilleformen blev firkantet. Det tegnes nu af os.

Trækket og palettens noter er ikke fastholdt i en prøve. ⌘K, tastning og en
debounce oven på hinanden gjorde den upålidelig, og en flakkende prøve siger
mindre end ingen.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 20:46:05 +02:00
Kristian WindandClaude Opus 5 43ad60a84a feat(notes): formateringen overlever, og monospace bliver til en terminal
Konverteringen læser nu Apple Noters HTML frem for at flade den ud med textutil.
Overskrifter, fed, kursiv, understreget, lister og monospace kommer med. En note er
ikke kun sine ord — et referat med overskrifter er ikke det samme referat som en
klump — og ti prøvenoter viste hvad det kostede.

Skrevet om til Python, fordi HTML skal læses og ikke gættes med regulære udtryk.

Den alvorligste fejl, prøvekørslen fandt: navn og krop blev hentet i hvert sit
kald, og Noter sorterer listen efter ændringsdato. Flyttede noget sig imellem de
to kald, fik en note én titel og en andens indhold. På ti noter er det synligt; på
tolv hundrede ville det være ubemærket rod. Der hentes nu ved id, og navn og krop
kommer i samme kald.

Sammenhængende monospace-linjer bliver til én blok. Hver linje i Noter er pakket
ind i sin egen div, hvis starttag tømmer den forrige — så mellem to tt-linjer kom
der altid en tom flush forbi, og hver linje i en terminaludskrift blev sin egen
blok.

Og i editoren ser en kodeblok ud som en terminal: mørk flade, monospace, sproget i
hjørnet, farver på strenge, tal, kommentarer, nøgleord og prompten. Monospace ER
kode i praksis, og en mørk flade siger det på afstand.

Farverne lægges på de blokke, markøren IKKE står i. At farve mens nogen skriver
betyder at bygge indholdet om ved hvert tastetryk og sætte markøren tilbage
bagefter — det virker ni gange ud af ti og flytter markøren midt i en sætning den
tiende.

Et API-token er ét ord, der er bredere end skærmen. Uden min-width: 0 skubbede en
sådan note hele listen bredere og klemte editoren sammen — på en note, man ikke
engang havde åben.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 20:31:10 +02:00
Kristian WindandClaude Opus 5 08fc355bf9 fix(tools): tre fejl, en prøvekørsel på ti noter fandt med det samme
Kodningen. textutil gætter på inddataets kodning og gætter forkert, så
"Ansøgning" kom ud som "Ansøgning". Det ser ud som én underlig note, indtil man
opdager, at det er hver eneste note med et dansk bogstav i.

Titlen stod to gange. Kroppen af en Apple-note begynder med notens egen titel, så
en overskrift ovenover gav den dobbelt overalt.

Og listerne var ikke lister. Apple skriver dem som tabulator, tegn, tabulator —
en liste for et menneske og ingenting for en Markdown-læser, så en huskeliste
ankom som en klump linjer. Punkttegnet er flerbyte, og BSD sed hakkede det midt
over, så hver linje begyndte med to halve tegn; perl med -CSD gør det rigtigt.

Scriptet tager nu et antal som andet argument, så en prøvekørsel er til at
gentage. Det er billigere at opdage den slags på ti noter end på tolv hundrede.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 19:37:52 +02:00
Kristian WindandClaude Opus 5 414a35eaba feat(notes): billeder og bilag følger med ind — og ud
Release / Build and publish image (push) Failing after 9m58s
Du spurgte, hvad der sker med bilag og indsatte billeder. Svaret var: de forsvandt
stille. Scriptet konverterede til ren tekst, og en note fra Apple Noter, der kommer
over som ord alene, er ikke den note — den er et referat af den.

En note kan nu bære filer, i samme tabel som opgaver og grupper allerede bruger.
Filerne er indholdsadresserede og deduplikeres på disken, upload-vejen er skrevet
og afprøvet, og oprydningen af forældreløse filer går allerede den tabel igennem.
En tabel mere ville være et alt-sammen mere.

Importen læser arkivet i to omgange. Alt, der ikke er Markdown, gemmes først, så
en note, der bliver læst, allerede har adresser at få sine links rettet til —
noter og billeder ligger i den rækkefølge, arkivet nu har dem, og ét gennemløb
ville efterlade halvdelen af henvisningerne pegende på ingenting.

Kun de filer, noten faktisk nævner. Et arkiv er én mappe til mange noter, og at
hænge hvert billede på hver note ville gøre hundrede noter til ti tusind bilag.

Hentningen af en note-fil gav 404, fordi adgangskontrollen ikke kendte noter som
forælder. Et billede midt i den tekst, der navngav det, svarede altså "findes
ikke". Den følger nu noten, som følger sit projekt — samme kæde som alt andet.

AppleScriptet gemmer nu hver vedhæftning og skriver en henvisning til den i noten.
Vedhæftninger uden fil bag sig — links til websider, kort — springes over frem for
at vælte kørslen.

Migration 0021.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 19:12:54 +02:00
Kristian WindandClaude Opus 5 11fdbefe1b feat(notes): hver note som en Markdown-fil, i et zip
Det er løftet, hele designet blev lagt til rette omkring: noten på disken ER den
fil, man ville eksportere. Der konverteres ingenting på vejen ud, så der er
ingenting, konverteringen kan tage fejl af — filerne åbner i Obsidian, i en
teksteditor, i hvad som helst.

To ting, en eksport kan tabe stille, og som prøven holder fast:

Et skråstreg i en titel ville lave en mappe på Mac og Linux og blive afvist på
Windows. De tegn, ingen af de tre vil have, bliver til bindestreger.

Og to noter med samme titel er to noter — et arkiv med den samme post to gange er
et arkiv, der har mistet den ene. Den anden får et nummer.

Fejler skrivningen midtvejs, stopper den frem for at rydde op. Svaret er allerede
på vej ud, så der er ingen statuskode tilbage at sende: et afkortet zip siger
selv, at det er afkortet, hvilket er bedre end en komplet fil med en note færre.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 19:03:25 +02:00
Kristian WindandClaude Opus 5 6e83291444 feat(notes): opgaven viser de noter, der nævner den — og har fået en adresse
Den anden vej af samme forbindelse. En note, der nævner en opgave, viste den; nu
viser opgaven noten. Uden det virker forbindelsen kun, hvis man tilfældigvis
sidder og læser noten, og det er den halvdel, ingen har brug for hjælp til.

Undervejs viste det sig, at en opgave slet ikke havde en adresse. Skuffen åbner
over den liste, man står på, og ændrer ingenting i URL'en — hvilket er rigtigt for
et klik og ubrugeligt for et link: man kan ikke sende nogen en opgave, og en note,
der nævner en, har ingen steder at sende dem hen. Min egen henvisningsform pegede
altså på noget, der ikke fandtes.

/opgave/{id} findes nu. Den åbner skuffen og træder til side — erstatter sig selv
med opgavens eget projekt, så man lander hvor opgaven bor, med den åben, og
tilbage-knappen ikke går gennem en side, der kun nogensinde viderestillede.

Prøven henter opgavens id fra API'et frem for fra adresselinjen, netop fordi den
ikke står der.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 18:59:34 +02:00
Kristian WindandClaude Opus 5 f6c60bdaf9 fix(notes): #garageristeriet og #GarageRisteriet er samme projekt
Release / Build and publish image (push) Failing after 9m58s
De var det ikke. Nøglen blev gemt præcis som skrevet og slået op på samme måde,
så en note, der nævnte et projekt med lille begyndelsesbogstav, dukkede simpelthen
ikke op på det — stille, og uden noget der antydede hvorfor.

Foldet på vej ind og foldet i opslaget, så sammenligningen bliver ved med at være
en nøjagtig lighed og bruger sit indeks. Havde man i stedet sænket kolonnen i
forespørgslen, skulle hele tabellen læses.

Kun projekter. En [[notetitel]] er en titel, ikke en nøgle: to noter, der hedder
"Møde" og "møde", er to noter, og at folde dem sammen ville slå noget sammen,
nogen havde holdt adskilt. Der er en prøve for begge dele.

Migrationen samler de rækker, der allerede er skrevet. Primærnøglen er
(note_id, kind, target_id), så en note, der skrev begge stavemåder, ville kollidere
med sig selv — derfor bygges sættet forfra med INSERT OR IGNORE frem for en
opdatering, der ville fejle på parret.

Fladen viser projektets egen stavemåde og ikke den foldede nøgle. En etiket, ingen
har skrevet, og som står hverken i noten eller på projektet, ville være værre end
ingen.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 18:48:00 +02:00
Kristian WindandClaude Opus 5 5e670119db feat(notes): titlen er første linje, og # foreslår projektet
Release / Build and publish image (push) Failing after 9m59s
Titlen var udledt én gang og derefter ladt i fred, så den gik i stå i det øjeblik
nogen skrev begyndelsen om — en note med overskriften "Ny note" stod i listen som
"fdgfgfgh", hvilket var, hvad dens første linje havde været nogle rettelser
tidligere. Den udledes nu ved hver gemning, og ingen kan sætte den ved siden af
teksten: kolonnen er en kopi af kroppen, ikke et felt derudover. Feltet er væk fra
både HTTP-fladen og MCP-værktøjerne, for et felt, serveren ignorerer, er et løfte,
den ikke kan holde.

Skriver man #Kaf, foreslås #Kaffebaren. Ordet læses fra den tekstknude, markøren
står i, og ikke fra hele dokumentet — en note kan nævne tyve projekter, og kun det
under markøren er ved at blive skrevet. Listen ejer piletasterne og retur, mens
den er åben, som enhver anden fuldførelse.

Og under feltet står nu, hvad teksten forstår: # projekt, [[en anden note]], ⌘B,
⌘U. Samme idé som linjen under hurtig tilføjelse — en note er det andet sted i
programmet, hvor det, man skriver, bliver læst for betydning.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 18:36:31 +02:00
Kristian WindandClaude Opus 5 3223989a9d feat(notes): første linje er titlen, resten er brødtekst
Release / Build and publish image (push) Failing after 9m57s
En ny note åbner på sin titel, og linjen efter er brødtekst — uden at man vælger
noget. Det er det, Apple Notes gør, og det er også hvad listen har brug for:
første linje bliver notens navn, så en note, der begynder i brødtekst, hedder sin
egen første sætning.

To ting undervejs. Reserven for en tom note blev aldrig brugt, fordi en tom tekst
ikke gengives som ingenting men som ét tomt afsnit — den prøves nu på teksten og
ikke på resultatet. Og Chromium svarer på retur med et bart <div>, som læses som
brødtekst men ikke er formateret som det: afstanden mellem afsnit forsvandt
stille i det øjeblik, man trykkede retur frem for at vælge Brødtekst i menuen.

Listen viste desuden den rå Markdown i forhåndsvisningen — "# Møde med Anders" og
"**fed**" — hvilket er præcis det, editoren blev bygget for at holde op med at
vise. Den viser nu sætningen frem for notationen.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 18:23:42 +02:00
Kristian WindandClaude Opus 5 f557d319ca feat(sidebar): de faste visninger kan sorteres, og rækkefølgen følger kontoen
Release / Build and publish image (push) Failing after 9m57s
I dag, Kommende, Venter og Indbakken kan trækkes i den rækkefølge, man vil have
dem. På kontoen frem for i browseren: sidebjælkens bredde er en kendsgerning om
skærmen, og svaret på en telefon er ikke svaret på en skærm — men hvilken
rækkefølge man vil have dem i, er en kendsgerning om personen.

En liste af nøgler frem for et tal pr. visning. Ukendte nøgler droppes og
manglende føjes til sidst, så en visning kan tilføjes eller fjernes uden at
strande nogens rækkefølge — hvilket allerede er sket én gang, da Færdige
forlod menuen.

To ting, prøverne fandt undervejs, og begge var mine:

Rækken som trækgreb gjorde alt indeni trækbart, og sidebjælken er allerede
modtager for opgaver — en opgave trukket til I dag holdt op med at lande, fordi
rækken havde taget gestussen. Et lille greb tager ingenting, før man griber det.
Og drop-håndteringen slugte begivenheden, før den tjekkede, om den overhovedet
var en menuflytning.

Migration 0019.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 17:19:46 +02:00
Kristian WindandClaude Opus 5 90bdf4b725 feat(look): en akse mere — hvordan det læses, ikke hvor lyst det er
Fire udseender ved siden af de fem temaer: Verdande, Rolig (serif, mere luft),
Tæt (flere rækker, mindre luft) og Terminal (én bredde til det hele). De ganges ud
til tyve udseender fra otte små blokke CSS frem for at kræve en blok hver.

To akser, fordi det er to spørgsmål. Den, der vil have en serif om natten, skal
ikke opgive den mørke palet for at få den — og prøven holder netop det fast: at
vælge en skrift må ikke trække temaet med sig.

Udseendet sættes før første optegning, ligesom temaet. Ellers bliver siden sat op
i én skrift og derefter i en anden, og det er et synligt spring.

Det er også det ærlige svar på at ligne noget andet. Standarden er bevidst
genkendelig — det var det, der blev bedt om — men et program med ét muligt
udseende har lånt en form, det ikke kan give tilbage. Med udseender er ligheden en
indstilling blandt flere frem for det eneste, det kan være.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 17:07:39 +02:00
Kristian WindandClaude Opus 5 1bbae698d5 feat(notes): en note deles ved at ligge i et projekt
Ikke en adgangsliste af sin egen. Projekter har allerede medlemmer, roller og
invitationer — alt sammen afprøvet, og alt sammen noget folk har lært her i
forvejen. En anden måde at give adgang på er et andet sted at gøre det forkert,
og de to ville være uenige den dag, den ene blev ændret.

To ting, prøven fandt, som begge var rigtige fejl.

Projektsiden viste kun noter, der NÆVNTE projektet, ikke dem, der lå i det — så
en note, man lige havde delt derind, dukkede ikke op. Nu vises begge slags.

Og noteoversigten viste kun dem uden projekt, så en note forsvandt fra listen i
det øjeblik, den blev delt. Det fik deling til at se ud som sletning. Listen viser
nu alt, man kan se: sine egne løse, og dem i de projekter, man er med i.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 17:03:28 +02:00
Kristian WindandClaude Opus 5 1b02eecdab feat(notes): rich text, med Apple Notes' egne stilarter
Man skriver nu i formateret tekst frem for i Markdown-kildekode. Titel,
Overskrift, Underrubrik, Brødtekst, Monotype, Blokcitat, punkt- og talliste, plus
fed, kursiv, understreget og gennemstreget — samme ordforråd som Apple Notes, for
det er det, en, der kommer derfra, allerede kan.

Filen under er stadig Markdown. Det er ikke et kompromis, nogen skal leve med i
stilhed: det er grunden til, at eksporten er til at stole på — filen, en note
bliver til, er den fil, den allerede var. Understregning er den ene undtagelse;
den findes ikke i noget Markdown-dialekt og gemmes som <u>, som enhver læser
sender uændret igennem. Det er bedre end at fjerne en stilart, kilden har.

contenteditable med execCommand. Den er forældet og også det eneste, alle browsere
er enige om; at skrive en markeringsmodel fra bunden er ugers arbejde og en ny
slags fejl netop dér, hvor tabt tekst er utilgiveligt.

Tre ting, prøverne fandt, som alle ville have kostet tekst:
Værktøjslinjen stjal markeringen, når man klikkede — knapperne tager ikke fokus
længere. Browseren skrev <span style="font-weight: normal"> for en ophævet fed,
hvilket en oversætter, der læser tags, taber; der bedes nu om tags, og styles
læses alligevel, fordi det ikke honoreres alle steder. Og markeringen med
Shift+Home tog hele dokumentet, ikke linjen.

Favoritter er stjernen på hver note. Noteoversigten har fået luft mod venstre
kant, og Gem og Slet er knapper frem for bare ord.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 16:58:53 +02:00
Kristian WindandClaude Opus 5 5757f06e7a feat(notes): man kan se hvad man skriver
Release / Build and publish image (push) Failing after 10m0s
Formatering vises, mens man skriver. Overskrifter, fed, kursiv, kode, citater,
punkter, #tags og [[links]].

Et spejl bag et gennemsigtigt tekstfelt — samme greb som hurtig tilføjelse bruger
til at vise, hvad parseren forstod. En contenteditable ville lade teksten style
direkte og ville også ødelægge fortryd, autokorrektur og IME. Det er en dårlig
byttehandel i det ene sted i programmet, hvor nogen skriver en halv time i træk.

Mærkerne bliver stående, dæmpede. Obsidian skjuler dem uden for den linje,
markøren står i; at dæmpe dem altid er en mindre idé, der ikke flytter tekst under
markøren mens den bevæger sig — og den overlever mødet med et tekstfelt.

Overskrifter skifter vægt og farve, aldrig størrelse: en større skrift ville fylde
flere linjer end tekstfeltet gør, og så skred alt nedenunder.

Hele risikoen ved grebet er, at de to lag skal flugte tegn for tegn. Prøven måler
det frem for at stole på det: skrifttype, linjehøjde, ombrydning, polstring og den
faktiske højde af de to lag med samme tekst i.

Undervejs: kursiv-passet slugte " og " efter et fed-udtryk, fordi den løse stjerne
stadig var synlig for det. Hvert pas læser nu en kopi, hvor de tidligere pas'
tegn er blændet ud.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 13:38:18 +02:00
Kristian WindandClaude Opus 5 5f04fe56bc feat(notes): note-værktøjer i MCP, og sidebjælken ryddet op
Claude kan nu skrive og søge i noter gennem MCP. Det er også svaret på Apple
Noter: der findes ingen eksport, der er værd at bruge, men en model, der kører på
maskinen, kan læse Notes.app og skrive det, den finder, ind herigennem — én note
ad gangen, med teksten som Markdown. En flytning, ingen skal bygge en parser til.

Værktøjet melder tilbage, hvilke links noten fik. En note, hvis #tag var stavet
forkert, ser ellers præcis ud som en, hvor det ramte, og i en chat ser ingen den
række, der blev skrevet.

Noter står nu for sig selv med en streg over. De er ikke en femte måde at se de
samme opgaver på, hvilket er hvad alt ovenfor er — de er den anden halvdel af
programmet, og i den række læste de som endnu et filter.

Færdige er ude af menuen og ligger under Indstillinger → Data ved siden af
papirkurven. Argumentet for at have den fremme var, at man ikke ved, hvilken
visning man lukkede noget i — det er sandt, og det er også et sjældent ærinde ved
siden af fire punkter, der læses hver dag. Den er der stadig, med en beskrivelse
af hvad den er til.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 13:33:54 +02:00
Kristian WindandClaude Opus 5 a2a39c4e17 feat(notes): en flade at skrive i, og noten dukker op i projektet
Release / Build and publish image (push) Failing after 9m55s
Tredje etape. En liste til venstre, noten til højre — ikke et gitter af kort: en
note læses langt oftere på sin første linje end i sin helhed, og en liste viser
tyve af dem, hvor et gitter viser seks.

Gemmes når man holder op med at skrive. En note er det ene sted i programmet, hvor
en times skrivning kan gå tabt på ét fejltryk, og en Gem-knap er en knap, der
findes for at blive glemt. Teksten holdes adskilt fra den note, den kom fra, så en
gemning, der lander sent, ikke kan overskrive det, der er skrevet siden.

Og udbetalingen: skriver man #GarageRisteriet i en note, står den på projektets
side. Ingen har filet den derhen — den dukker op, fordi nogen nævnte projektet
mens de skrev. Slået op på navnet og ikke på et id, for et #tag skrives i hånden,
og et menneske skriver navnet.

Markdown i et almindeligt tekstfelt indtil videre. Live preview, hvor mærkerne kun
vises i den linje markøren står i, er næste stykke og det dyre; det her er den
form, den skal sidde inde i, og alt andet er allerede sandt uden den.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 13:07:49 +02:00
Kristian WindandClaude Opus 5 fbfc98e217 feat(notes): noter kan nås — listet, søgt og fundet baglæns
Anden etape. Handlere, ruter og OpenAPI.

Adgang følger projektet, for det er der, noten bor: en note i et delt projekt kan
læses af alle, der kan læse projektet, og ændres af en redaktør. En note uden
projekt er sin forfatters alene — samme svar som en opgave i indbakken får, nået
frem til på samme måde frem for ved en regel, noter fandt på for sig selv.

Listning og søgning er samme endepunkt, fordi de svarer på det samme spørgsmål
med forskellige ord. Filtreringen sker EFTER indekset: FTS5 svarer om tekst, og
at lære indekset om roller ville lægge en adgangsregel et sted, ingen ville kigge
den dag den var forkert. Der er en prøve for, at en fremmeds søgning ikke finder
noget, og at en fremmed får 404 og ikke 403 — så et id ikke kan afprøves.

Den baglæns rute hænger på det, der spørges om: et opgave-id giver hver note, der
nævner den; et projektnavn giver hver note med det #tag.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 13:00:41 +02:00
Kristian WindandClaude Opus 5 98ea08b0bb feat(notes): fundamentet — Markdown, søgning og links begge veje
Release / Build and publish image (push) Failing after 10m3s
Første etape af noter. Tabel, CRUD, fuldtekstsøgning og de links, en note bærer.

Markdown i en kolonne, ikke rich text og ikke et format af vores eget. Eksport
var et krav, før der var skrevet en linje, og er den gemte form den eksporterede
form, er der ingen konvertering tilbage at tabe noget i — noten kan læses af
enhver anden editor den dag programmet ikke findes.

Et #tag i en note er DET SAMME tag som i en opgave. Ikke en anden slags, der
ligner. Det er hele svaret på "tag en note og se den i projektet": referencen er
den samme, så der ikke opstår to systemer, brugeren selv skal holde i sync.

Links gemmes som rækker, udledt af teksten ved hver gemning. Teksten er stadig
sandheden; tabellen er et indeks over den. Uden det ville "hvad peger på denne
opgave" betyde en gennemsøgning af al prosa, og den rude ville være for langsom
til at lade stå åben. De rives ned og bygges op frem for at blive sammenlignet:
en sammenligning er en anden beskrivelse af samme kendsgerning, og den kan komme
til at være uenig med teksten.

Noter ligger under projekter og arver deling, roller, papirkurv og historik. En
mappestruktur ved siden af ville være ét hierarki for meget.

MatchExpr fik navngivne kolonner. Den var skrevet til opgavernes og gav
"no such column" mod noternes — hvilket i det mindste er højlydt.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 12:47:50 +02:00
Kristian WindandClaude Opus 5 c44099cac0 feat(site): vælg lys eller mørk, og skærmbilledet følger med
Siden fulgte systemet og kunne ikke andet. Nu er der en knap, valget huskes, og
skærmbilledet øverst skifter med — der ligger både en mørk og en lys udgave af
det samme, og det er kejtet at vise et mørkt skærmbillede på en lys side.

Farverne står to gange med vilje. Mediaforespørgslen er hvad en førstegangsbesøgende
får; attributten er hvad en, der har valgt, får — og den skal vinde begge veje, både
et valgt lyst tema på et mørkt system og omvendt. Én regel kan ikke gøre begge dele.

Begge skærmbilleder ligger i markuppen og skjules med CSS, så ingen af dem skal
hentes i det øjeblik nogen trykker på knappen.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 12:36:11 +02:00
Kristian WindandClaude Opus 5 86ec0ce2d9 fix(imap): markøren er vores, ikke serverens
Samme mail blev en opgave hver tiende minut, og markøren stod stille på 1271.
Målt i produktion: en manuel synk lavede én opgave og flyttede ikke last_uid en
eneste plads.

Årsagen er ikke vores regnestykke, men hvad søgningen svarer. Vi beder om
1272:* — og en rigtig server læser det som 0:1272 og sender hele postkassen
tilbage. Alt, hvad den finder, ligger så under markøren, og markøren kan derfor
aldrig komme videre.

Uid'er ved eller under markøren frasorteres nu hos os. Det koster ét gennemløb
af nogle hundrede tal og afhænger ikke af nogen andens læsning af specifikationen.

De to tidligere forsøg var ikke forkerte, de var bare ikke årsagen: at tage
uid'en fra søgningen frem for fra fetch-svaret, og at give hver postkasse sin
egen lås, var begge reelle fejl. Denne er den, der fik den til at gentage sig.

Prøven læser en postkasse med markøren over alt i den og kræver nul tilbage —
og gentager det for hver enkelt besked.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 12:33:30 +02:00
Kristian WindandClaude Opus 5 62498c2765 feat(sidebar): tallet ved et projekt er opgaver, og folden har en tast
"Hvorfor står der 2 ud for et tomt projekt?" Fordi tallet var member_count — hvor
mange mennesker. Den samme grå kolonne betød tre ting: personer ved projekter,
projekter ved grupper, opgaver ved etiketter. I en opgaveapp læses et tal ved
siden af et projekt som opgaver, og det gjorde manden, der har skrevet appen.

Nu er det åbne opgaver, og det vises når der er noget tilbage frem for når
projektet er delt — som var den gamle betingelse, og grunden til at netop de tre
delte havde et tal. Hvem der ellers er med står på projektets egen side.

Tællingen hentes fra serveren igen, når opgaver flytter sig. Det ville være
hurtigere at lægge til og trække fra lokalt, og det ville være forkert inden for
en dag: en opgave flyttet mellem projekter, en lukket i en anden fane, en
gentagende der lukker og åbner sig selv.

Og sidebjælken kunne allerede foldes — der var bare kun en knap med et rektangel
med en streg i, og det er ikke et ord. Nu gør ⌘B det samme, knappen siger det, og
genvejssiden nævner det.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 12:28:43 +02:00
Kristian WindandClaude Opus 5 997dffe01c docs: skriv det, der blev bygget, og ret det, der ikke passer længere
To nye sider. Postkasser: at stjerne i Gmail eller flage i iCloud og Fastmail
bliver til opgaver, at hver postkasse hører til den, der forbandt den, at
app-kodeord er noget andet end ens rigtige, og at Office 365 ikke kan nås ad den
vej — sidstnævnte skrevet ned frem for at blive opdaget ved at prøve. Og at logge
ind: kodeord, totrinskode og passkeys, med hvorfor en passkey ikke kan laves på
en bar IP-adresse.

Konfigurationen manglede VERDANDE_SECRET_KEY, som er den, det koster mest ikke at
vide noget om: mister man nøglen, skal postkasserne forbindes forfra. Begrundelsen
står med — det er kopierne, ikke serveren, der er problemet, for de kan hentes
gennem fladen.

Og genvejsafsnittet i quick-add beskrev q, t og u, som ikke findes længere. En
dokumentation, der lover taster, der ikke gør noget, er værre end ingen: man
prøver, der sker intet, og så tror man appen er i stykker.

Bygget med --strict; alle krydshenvisninger rammer.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 11:25:54 +02:00
Kristian WindandClaude Opus 5 d02d8edfe9 feat(site): klik et skærmbillede og se det i læsbar størrelse
Release / Build and publish image (push) Failing after 10m5s
Et skærmbillede af en brugerflade siger ingenting ved 281 pixels, og det er der,
de fleste af dem sidder i gitteret. Et klik åbner nu det samme billede omkring
tre en halv gang større.

<dialog> frem for et selvbygget lag: Escape lukker, fokus bliver inde i den,
resten af siden bliver inert — tre ting browseren gør gratis og som et selvbygget
overlay skal have kode for hver af. Billedet får rolle og tabindex, så det også
kan åbnes fra tastaturet; en kontrol, kun en mus kan nå, er ikke en kontrol alle
har.

To ting rettet undervejs, som først kunne ses ved at kigge: billedet fik lov at
blive højere end skærmen, så billedtekst og lukkeknap røg op over kanten — og
lukkeknappen er den synlige vej ud. Og dialogen fyldte hele bredden i stedet for
at slutte om billedet, så skærmbilledet stod op ad venstre kant med knappen ude i
det modsatte hjørne. Nu er den fit-content og centreret, og alt er inden for
vinduet ned til en telefonbredde.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 11:14:13 +02:00
Kristian WindandClaude Opus 5 134fe8918e chore: hold overleveringen uden for repoet
Den navngiver panelet, server-id'et og produktionsadressen — et kort over én
persons opsætning, og det hører ikke til i et offentligt repo. Ingen adgangskoder,
men et server-id og en panel-adresse er præcis det, et lækket token skal bruge for
at være farligt.

Filen er fjernet fra hele historikken, ikke kun fra spidsen: at slette den i en ny
commit ville have efterladt den læselig i alle de gamle. Den ligger stadig på
disken, for det er sådan en session afleverer til den næste.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 11:08:08 +02:00
Kristian WindandClaude Opus 5 3fd16f71df fix(reminders): en påmindelse, ingen modtager, er ikke en påmindelse
Påmindelser blev sendt i appen og på mail. Har man fanen lukket og ingen
mailserver sat op — hvilket er præcis det, docker-linjen i dokumentationen giver
— kom der ingenting, mens indstillingssiden sagde at notifikationer var slået til
for enheden.

Push-halvdelen var aldrig hægtet på. Notifikationer fra andres handlinger gik ud;
kun de tidsstyrede gjorde ikke, og det er dem, man har slået til for at slippe for
at holde øje selv. Jobbet har nu en vej til at sende, leveret af HTTP-laget, som
ejer VAPID-nøglerne og listen over enheder — samme mønster som synkroniseringerne.

Prøven ser efter, at den når frem, og at den kun når frem én gang: en påmindelse,
der ankommer ved hver fejning, er værre end en, der ikke ankommer.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 11:00:48 +02:00
Kristian WindandClaude Opus 5 37605ba684 feat(site): landingssiden siger hvad der findes, og vejer en femtedel
Skærmbillederne lå som PNG i fuld skærmopløsning — 2,1 MB for syv billeder på en
side, der viser dem under 900 pixels bred. Som WebP i dobbelt visningsbredde
fylder de 424 kB. Samme billeder, samme skarphed, en femtedel af vægten.

Indholdet fangede aldrig op på det, der blev bygget: postkasser over IMAP ved
siden af Gmail, passkeys, at man kan skrive hvor som helst for at fange noget, at
hemmeligheder ligger krypteret med nøglen uden for databasen, og historikken.
Ni felter nu, hvilket også fylder gitteret ud i stedet for at efterlade et tomt.

Og en fejl, jeg selv lavede undervejs: jeg lagde en description-meta ind, fordi
min søgning ikke så den, der lå i forvejen — den var delt over flere linjer.
Dubletten er væk, og den oprindelige er opdateret i stedet.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 10:52:20 +02:00
Kristian WindandClaude Opus 5 6e941f9928 fix(mailboxes): to kørsler af samme postkasse må ikke læse den samme markør
Fyrre kopier af én mail ankom i samme sekund. Én kørsel kan højst lave femogtyve,
så der var to i gang: et tryk på "Hent nu" mens ti-minutters-fejningen allerede
var inde i den samme postkasse. Begge læste markøren som den stod, begge hentede
de samme beskeder, begge lavede opgaver ud af dem.

Markør-rettelsen fra i går var altså rigtig, men ikke nok — den virker, når der
kun er én læser. Rækken læses nu igen under låsen, og låsen holdes hele vejen
igennem: slap man den, før markøren var skrevet, ville hullet stå åbent præcis
der, hvor det skulle lukkes.

Et kort pr. postkasse frem for én lås. To menneskers postkasser har intet at sige
hinanden, og én fælles lås ville gøre én langsom vært til alles problem.

Prøven kører to kørsler oven i hinanden med vilje og tæller, hvor mange der er
inde ad gangen — en lås er kun værd at have, hvis noget prøver døren. Den holder
også øje med, at to forskellige postkasser ikke venter på hinanden.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-19 08:57:59 +02:00
Kristian WindandClaude Opus 5 993822f6fd fix(panel): bed om en planlagt genstart, ikke en der dør med afsenderen
Release / Build and publish image (push) Failing after 10m6s
Knappen tog produktionen ned og lod den ligge. Panelets almindelige restart
udfører arbejdet mens forespørgslen står åben: den stopper containeren — hvilket
slår denne proces ihjel — og panelet, der ser afsenderen forsvinde, når aldrig
til den halvdel, der starter igen. Fire minutters nedetid den 18. august, og det
kan ikke lappes herfra: enhver knap, der synkront beder om sin egen død, har
samme form.

safe-restart planlægges i stedet. Panelet svarer "Restart scheduled" og gør
arbejdet bagefter for egen regning, så intet afhænger af, at denne proces stadig
er i live til at høre svaret.

Prøven hang bagefter, og det var prøvens skyld: dens falske panel læste aldrig
forespørgslens krop, og der er en nu. En server, der ikke læser hvad den fik,
opdager aldrig at afsenderen har lagt på — så httptest ventede på en handler, der
aldrig vendte tilbage, og hele binæren løb tør for tid i stedet for den ene
forespørgsel.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-18 23:15:10 +02:00
Kristian WindandClaude Opus 5 b0f78d1549 feat(gmail): Gmail flytter ind i postkasse-tabellen
Ét sted at spørge "hvilke postkasser har den her person". Gmails tokens, udløser,
etiket, set-liste og målprojekt ligger nu som en række i mailboxes ved siden af
dem, der læses over IMAP, og baggrundsjobbet fejer én gang i stedet for to og
skifter vej efter rækkens kind.

PKCE-værdierne bliver i user_settings. De hører til ét forsøg på at logge ind,
ikke til en postkasse, og de skal væk igen når forsøget er ovre.

Målprojektet fik sin egen kolonne, som IMAP-postkasser også får glæde af — en
arbejdspostkasse, der filer i arbejdsprojektet, er det åbenlyse at ville have.
Ikke en fremmednøgle: sletter man projektet, skal destinationen stille holde op
med at gælde, ikke afvise sletningen eller tage postkassen med sig.

Flytningen kunne først køre nu. Den tømmer indstillingsrækken, og havde den ligget
i v0.14.0, mens handlerne stadig læste den, var en fungerende postkasse blevet
koblet fra. Værdierne kopieres som chiffertekst — begge tabeller forsegler med
samme nøgle — så migrationen behøver ikke nøglen, hvilket en migration heller
ikke har noget at gøre med at holde.

Der er en prøve for præcis dét: en Gmail forbundet før flytningen, med et forseglet
token skrevet som indstillingstabellen skrev dem, kommer ud på den anden side med
token, udløser, adresse, målprojekt og set-liste i behold.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-18 22:59:40 +02:00
Kristian WindandClaude Opus 5 f3fecbd0b4 fix(imap): samme mail blev en ny opgave hver gang, og uddraget var rå MIME
To fejl fra ét skærmbillede.

Markøren stod stille. Den blev sat ud fra uid'en i fetch-svaret, og den kommer
ikke fra alle servere — så next-run læste fra nul hver gang. Den sættes nu ud fra
hvad SØGNINGEN fandt, når kørslen kom hele vejen igennem: så er alt op til det
punkt betragtet, uanset hvad fetch-svaret bar med. En afbrudt kørsel flytter kun
til den sidste, den faktisk lavede en opgave af. Prøven mod hukommelsesserveren
ville aldrig have fanget det, for den svarer altid med uid.

Og uddraget hentede BODY[TEXT], som ved multipart er grænser og dele-hoveder og
det hele — derfor stod der "--=_be71cb39… Content-Transfer-Encoding: 8bit" foran
teksten i hver eneste opgave. Nu hentes hele beskeden og MIME-strukturen gås
igennem: text/plain foretrækkes, HTML er reserven med mærkerne pillet af. Groft,
for det er linjen under en overskrift og ikke en gengivelsesmotor.

Multipart-prøven er sat på prøve: med den gamle hentning fejler den med præcis
den tekst, skærmbilledet viste.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-18 22:44:00 +02:00
Kristian WindandClaude Opus 5 30adc69cb5 feat(mailboxes): plads til Gmails udløser og dens set-liste
Kolonnerne, ikke flytningen. Gmail-handlerne læser stadig user_settings, og en
migration, der tømte den tabel før de var skrevet om, ville koble en fungerende
postkasse fra. Det additive først; flytningen når koden på den anden side er klar
til den, og den ligger som PENDING_gmail_move.sql.txt uden for .sql-mønstret,
så den ikke kan køre ved et uheld.

Udløseren fik sin egen kolonne frem for at blive pakket ind i label med et kolon.
En kolonne, der rummer to ting adskilt af et tegn, er en kolonne, nogen læser
forkert senere — og en etiket med kolon i navnet ville gøre det på første dag.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-18 22:38:45 +02:00
Kristian WindandClaude Opus 5 c9dfffd95a feat(mailboxes): postkasser kan forbindes fra fladen
Release / Build and publish image (push) Failing after 9m59s
Panelet under Integrationer: tilføj, hent nu, kobl fra. Sit eget frem for en
tilstand mere inde i Gmail-panelet, for de to er ikke det samme i forskellige
klæder — det ene er en app, man logger ind igennem, det andet et kodeord, man
selv holder.

Feltet hedder IMAP-server og ikke Server: der stod i forvejen Server på CalDAV
længere nede, og to felter med samme navn på én side er tvetydige også for en
skærmlæser. Testen fandt det, men det var ikke kun testens problem.

To vagter fangede mangler undervejs, og begge havde ret: mailbox_failed skulle
have en tekst i fladen, ellers ville nogen få serverens engelske logsprog at se,
og de tre ruter skulle i openapi.yaml. Der er nu et Mailbox-skema samme sted, som
siger hvad der ikke kommer tilbage — kodeordet og eventuelle tokens.

E2E-prøven forbinder til noget, der ikke lytter, og ser efter værtens egen
afvisning på skærmen. Den er samtidig prøven på at 422 var det rigtige valg: var
det stadig 502, ville et mellemled kunne kaste beskeden væk, og der ville stå
"noget gik galt".

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-18 22:22:31 +02:00
Kristian WindandClaude Opus 5 6b4ef5b2f2 feat(imap): postkasser kan læses, og det er bevist mod en rigtig server
Handlere til at forbinde, koble fra og hente nu, plus baggrundssynk hvert tiende
minut ved siden af Gmails. Forbindelsen prøves FØR den gemmes: en postkasse, der
blev taget imod og så fejler stille hvert tiende minut, er værre end en, der blev
afvist ved døren, mens man stadig sidder med kodeordet i hånden.

Ingen "set"-liste til deduplikering som Gmail har. Uid'er er monotone, så
last_uid er hele svaret — og den skrives også, når budgettet løb ud, for en
kørsel, der lavede opgaver og glemte hvor langt den nåede, laver dem alle igen
næste gang.

Rundturen køres nu mod go-imap's egen server i processen: flaget post kommer med,
uflaget gør ikke, og fortsættelsen fra et uid henter kun det nyere. De hidtidige
prøver dækkede de rene dele og ville alle have bestået med selve hentningen i
stykker.

Testen kræver et selvsigneret certifikat, og klienten nægter dem. Account.RootCAs
lader en test TILFØJE tillid; der er stadig ingen måde at fjerne den på. Forskellen
er en søm og ikke et hul — en prøve viser, at samme server afvises uden.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-18 22:15:50 +02:00
Kristian WindandClaude Opus 5 1e59886d86 feat(mailboxes): en tabel, så I kan have hver sin postkasse — og flere hver
Gmail lå i user_settings under en scope. Den tabel har user_id som hele
primærnøglen, så den rummer præcis én scope pr. person, og dens upsert skriver
scope-navnet om ved konflikt: at lægge IMAP ved siden af ville have slettet
Gmail-forbindelsen uden at sige det. Migration 0011 advarer ordret mod netop det,
og advarslen var tæt på at blive gået i for anden gang.

Formen var forkert alligevel. Man har postkasser, flertal — en arbejds-Gmail og
en privat iCloud er det almindelige tilfælde, ikke det sære. En række hver siger
det. Og den svarer på spørgsmålet, der satte det i gang: hver postkasse hører til
en bruger, så to mennesker på samme instans forbinder hver sin og ser kun sin
egen. Det eneste, der bliver fælles, er Googles OAuth-klient, og det er rigtigt:
den er appen, ikke kontoen.

Kodeord og tokens skrives forseglet. Forseglingen i notifications.go dækker kun
indstillingstabellerne, så denne tabels metoder kalder den i hånden — værd at
vide, før der kommer en tredje hemmelig kolonne.

Gmail bliver hvor den er indtil videre. Den virker, og at lægge en flytning af en
kørende forbindelse i samme ombæring som en ny integration gør begge dele sværere
at efterprøve. Skemaet har allerede plads til den.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-08-18 22:10:20 +02:00
155 changed files with 17608 additions and 790 deletions
+9
View File
@@ -60,3 +60,12 @@ node_modules.nosync/
# accident — 8.9 MB of macOS binary — and `go build ./tools/shots` without -o puts
# it right back.
/shots
# The working handover. It names the panel, the server id and the production
# address — a map of one person's setup, which does not belong in a public repo.
# Kept on disk because it is how a session hands over to the next one.
HANDOFF.md
NOTES-PLAN.md
# Agenternes arbejdstræer. Et git-repo inde i et git-repo, som ikke hører til her.
.claude/worktrees/
+23 -12
View File
@@ -6,7 +6,7 @@
</p>
<p align="center">
<img src="site/screenshots/today.png" alt="The Today view: projects, filters and labels in the sidebar, beside the tasks due today." width="900">
<img src="docs/screenshots/today.png" alt="The Today view: projects, filters and labels in the sidebar, beside the tasks due today." width="900">
</p>
---
@@ -35,6 +35,11 @@ mail-to-task address, and an MCP endpoint so Claude can work your task list dire
in Firma, labelled regnskab. Danish and English, mixed freely in one line.
- **Projects, sections, sub-tasks, labels, saved filters.** List, board and calendar
views, and foldable groups over the projects in the sidebar.
- **Notes, beside the work.** Rich text that is Markdown underneath — headings,
lists, quotes, fenced code. A note is shared by filing it in a project;
`#project` and `[[another note]]` are links, and a task shows the notes that
mention it. A zip of Markdown goes in — Obsidian, Bear, or Apple Notes through
the exporter in [`tools/`](tools/) — and comes back out the same shape.
- **Drag where it means something.** Reorder a list or a board, file a project under
a group, drop a task on another project, or drop it on another day.
- **Sharing.** Owner, editor and viewer roles. There is no open registration:
@@ -90,21 +95,24 @@ Apple Reminders, forward mail into it, and add it to Claude as a connector.
| Project templates | Done, tested |
| Nightly backups with rotation | Done, tested |
| Comments, attachments, notifications, Web Push | Built |
| Notes: rich text, Markdown on disk, links to projects, notes and tasks | Done, tested |
| Notes in and out: a zip of Markdown, both directions | Done, tested |
| Todoist CSV import and export, account export | Done, tested |
| MCP server for Claude | Done, tested |
| CalDAV server (two-way VTODO) | Done, tested |
| Mail-to-task | Done, tested |
| AI layer (Anthropic/OpenAI/Google/local) | Built, degrades when unconfigured |
| Gmail: OAuth2 with PKCE, polling, one-way to tasks | Done, tested |
| Google Calendar: read-only, laid over the task calendar | Built, never run against Google |
| Update notice for administrators | Done, tested |
| Personal API tokens | Done, tested |
| OpenAPI 3.1 spec, checked against the router | Done, tested |
| End-to-end smoke tests (Playwright) | Done |
| Documentation, landing page, licence | Done |
**Not finished:** Web Push, the AI providers and the Gmail API calls are
implemented against their specifications and have only ever been exercised there,
not against live services. Everything else has an interface: opening a task shows
**Not finished:** Web Push, the AI providers, and the Gmail and Google Calendar
API calls are implemented against their specifications and have only ever been
exercised there, not against live services. Everything else has an interface: opening a task shows
its description, sub-tasks, comments, files and reminders, and the settings pages
cover the account, notifications, integrations, AI, API tokens, import, export and
templates.
@@ -116,17 +124,20 @@ accented `o` and a Danish-first app that cannot do this is broken for its own us
| | |
|---|---|
| ![Board view](site/screenshots/board.png) | ![Light theme](site/screenshots/today-light.png) |
| ![Board view](docs/screenshots/board.png) | ![Light theme](docs/screenshots/today-light.png) |
| Board view — one column per section. | The light theme. |
| ![A week on Upcoming](site/screenshots/week.png) | ![A task, opened](site/screenshots/detail.png) |
| A week at a time, so a task can be dragged across a month boundary. | A task, opened. |
| ![The five themes](site/screenshots/themes.png) | ![Instance history](site/screenshots/history.png) |
| Five themes, all measured for contrast. | What has been done on the server. |
| ![A week on Upcoming](docs/screenshots/week.png) | ![A task, opened](docs/screenshots/detail.png) |
| A week at a time, so a task can be dragged across a month boundary. | A task, opened: description, priority, labels, sub-tasks and its files. |
| ![Notes](docs/screenshots/notes.png) | ![The same note as Markdown](docs/screenshots/notes-source.png) |
| Notes, filed in a project to share them. | The same note as its source — which is what the export writes to disk. |
| ![Themes and looks](docs/screenshots/themes.png) | ![Instance history](docs/screenshots/history.png) |
| Five themes and four looks: how bright, and how it reads. | What has been done on the server. |
Regenerate these with `node scripts/generate-screenshots.js` from `web/`. It starts
the real binary, creates its own account, fills it with demo data and photographs
eleven surfaces — nothing to set up and nothing to remember. (`go run ./tools/shots`
still works, and is the way to photograph an instance that is already running.)
the real binary, creates its own account and a second one to share with, fills them
with demo data and photographs fourteen surfaces — nothing to set up and nothing to
remember. (`go run ./tools/shots` still works, and is the way to photograph an
instance that is already running.)
## Running it
+21 -6
View File
@@ -10,13 +10,28 @@ reply within a week.
- **Passwords** are Argon2id with the parameters recorded in the hash, so they can
be raised later without locking anyone out.
- **Every token** — session, invite, reset, API, calendar feed, mail address — is
stored only as a SHA-256 hash. A database dump contains nothing that can be
presented.
- **Session, invite, reset and API tokens** are stored only as a SHA-256 hash. A
database dump contains nothing that can be presented.
- **The calendar feed address and the mail-to-task address are not hashed.** They
are looked up by their exact value, and they are shown in the interface more
than once, which a hash cannot do. Anybody holding a backup file therefore holds
those two URLs: the feed reads that person's tasks without a login, and the mail
address files tasks into their inbox. Rotating either from **Settings →
Integrations** invalidates the old one. This is written down rather than fixed
because fixing it means the address can only ever be shown once, which is a
decision about how the feature works and not only about how it is stored.
- **Mailbox passwords, OAuth tokens, TOTP secrets and AI keys** are sealed with
AES-GCM under a key kept outside the database — see `internal/secret`. That is
what makes a stray copy of the database less than a stray copy of everything:
the backup does not carry the key.
- **Sessions** are `httpOnly`, `SameSite=Lax`, and `__Host-` prefixed over HTTPS.
- **Uploads** are stored content-addressed by hash and always served as
`application/octet-stream` with `nosniff`, so an uploaded SVG cannot run script
on this origin.
- **Uploads** are stored content-addressed by hash and served as
`application/octet-stream` with `nosniff` — with one exception, added so that a
note can show the pictures in it: a short allowlist of raster image types is
served inline as its own type. The list is named one format at a time rather
than matched on an `image/` prefix, because the prefix would let `image/svg+xml`
through, and an SVG is a document that can run script. `nosniff` stays, so the
browser must read the file as that image or not at all.
- **The filter language** compiles to parameterised SQL. No typed text is ever
concatenated into a statement.
- **Permission checks** are route middleware, and anything you may not see answers
+4
View File
@@ -74,6 +74,10 @@ func run() error {
defer stopJobs()
runner := jobs.New(cfg, db, log, api.Mail(), api.Hub())
runner.SyncGmail = api.SyncGmail
runner.SyncMailbox = api.SyncMailbox
runner.SyncCalendars = api.SyncCalendars
runner.Push = api.PushToUser
runner.SendBeacon = api.SendBeacon
runner.Start(jobCtx)
srv := &http.Server{
+44 -7
View File
@@ -1,13 +1,14 @@
# Calendar and CalDAV
Two ways to get your tasks into a calendar, and they are not the same thing.
Two ways to get your tasks into a calendar, one way to get a calendar into
verdande, and none of the three is the same thing.
| | [Feed](#subscribing-to-a-feed) | [CalDAV](#connecting-a-caldav-client) |
|---|---|---|
| Direction | Read only | Two-way |
| Works with | Anything that subscribes to a URL | Apple Reminders, Thunderbird, DAVx⁵ |
| You can tick things off | No | Yes |
| Setup | Paste a URL | Username and token |
| | [Feed](#subscribing-to-a-feed) | [CalDAV](#connecting-a-caldav-client) | [Google Calendar](#showing-a-google-calendar-in-verdande) |
|---|---|---|---|
| Direction | Tasks out, read only | Tasks out, two-way | Events **in**, read only |
| Works with | Anything that subscribes to a URL | Apple Reminders, Thunderbird, DAVx⁵ | A Google account |
| You can tick things off | No | Yes | No — an event is not a task |
| Setup | Paste a URL | Username and token | Connect, then pick calendars |
## Subscribing to a feed
@@ -39,6 +40,42 @@ most people actually keep.
**Settings → Calendar feed → Rotate** issues a new one and breaks the old
immediately.
## Showing a Google calendar in verdande
The two sections above send your tasks *out*. This is the other direction: your
Google calendar drawn underneath them, so one screen answers "what does this day
already have in it".
**Settings → Integrations → Google Calendar → Connect**, then tick the calendars
you want and press Save. They appear in **Calendar** in the sidebar, in Google's
own colour for each one, beside your own tasks with due dates.
It needs the same OAuth client Gmail uses, with the Calendar API enabled and the
second redirect URI registered — see [Configuration](configuration.md#google).
!!! warning "Read only"
verdande never writes to your Google calendar. An event cannot be dragged,
edited or ticked off here; clicking one opens it in Google, which is where it
can be changed. Your own tasks in the same grid still drag to another day as
they always have.
Writing would need the `calendar.events` scope and an answer to what should
happen when both sides moved the same meeting — which is a synchronisation
model rather than a scope, and it is deliberately not built yet.
!!! note "The Internal client applies here too"
An OAuth client registered as *Internal* in a Google Workspace can only be
used by accounts in that workspace, calendars included. A personal
`@gmail.com` address is refused with `org_internal`. For one personal account
the shorter road is the calendar's own secret address: in Google Calendar,
**Settings → the calendar → Integrate calendar → Secret address in iCal
format**. Nothing in verdande subscribes to one yet — it is the next piece of
this feature.
Events are kept for ninety days back and a year forward, refreshed every fifteen
minutes and whenever you press **Fetch now**. Paging the grid past that window
says so rather than showing an empty month.
## Connecting a CalDAV client
This is the one where ticking something off in Apple Reminders ticks it off here.
+62 -9
View File
@@ -39,12 +39,24 @@ Any Go duration: `720h`, `30m`, `45s`.
yourself, password resets are written to the log, and reminders arrive in the
app and as Web Push rather than by email.
## Gmail
## Google
Only needed if anyone will connect a mailbox. Register an OAuth client in Google
Cloud (**APIs & Services → Credentials → OAuth client ID → Web application**) with
the redirect URI `https://your-address/oauth/gmail/callback`, enable the Gmail API,
and add the `gmail.readonly` scope.
Only needed if anyone will connect a Gmail mailbox or a Google calendar. One
registration covers both: register an OAuth client in Google Cloud
(**APIs & Services → Credentials → OAuth client ID → Web application**) and add
**both** redirect URIs:
```
https://your-address/oauth/gmail/callback
https://your-address/oauth/calendar/callback
```
Then enable the APIs and scopes for whichever you want:
| For | Enable | Scope |
|---|---|---|
| Mailboxes | Gmail API | `gmail.readonly` |
| Calendar | Google Calendar API | `calendar.readonly` |
| Variable | What it does |
|---|---|
@@ -52,8 +64,45 @@ and add the `gmail.readonly` scope.
| `VERDANDE_GMAIL_CLIENT_SECRET` | The client secret. |
One registration serves every user on the instance; each person authorises their
own mailbox from **Settings → Gmail**. The redirect URI is derived from
`VERDANDE_BASE_URL`, so it cannot drift out of step with what you registered.
own mailbox from **Settings → Integrations → Gmail** and their own calendar from
**Settings → Integrations → Google Calendar**. Both redirect URIs are derived from
`VERDANDE_BASE_URL`, so they cannot drift out of step with what you registered.
!!! note "Two connections, two grants"
Connecting a calendar does not disturb a working Gmail connection, and
disconnecting one does not take the other with it. Google issues a refresh
token per authorisation and keeps up to a hundred of them live per account,
each carrying only the scopes it was granted with.
## Secrets
| Variable | Default | What it does |
|---|---|---|
| `VERDANDE_SECRET_KEY` | — | Encrypts the mail tokens and mailbox passwords in the database. 32 bytes of base64. |
Mail tokens and [mailbox](mailboxes.md) passwords are encrypted at rest, and the
key does not live in the database. That is the whole point: **backups can be
downloaded through the interface**, so a copy that ends up somewhere it should
not would otherwise open your mail.
Leave the variable empty and verdande writes a key to `secret.key` in the data
directory on first start, mode `0600`. Backups are a `VACUUM INTO` of the
database alone, so the key file is not in them.
Generate one yourself if you would rather keep it out of the volume entirely:
```
openssl rand -base64 32
```
!!! warning "Lose the key and the mailboxes must be reconnected"
Nothing else is lost — tasks, projects, files and history are not encrypted —
but a token that cannot be decrypted cannot be used, and there is no way back
from a key that is gone. If you restore a database somewhere new, bring
`secret.key` with it, or set `VERDANDE_SECRET_KEY` to the same value.
A database written before there was a key is read as it stands and encrypted the
next time each value is written, so turning this on costs nobody a reconnection.
## Update checks
@@ -64,9 +113,13 @@ own mailbox from **Settings → Gmail**. The redirect URI is derived from
| `VERDANDE_PANEL_TOKEN` | — | An API token from that panel, belonging to somebody with control of this server. |
| `VERDANDE_PANEL_SERVER_ID` | — | Which server to restart — the id in the panel's URL. |
All three together let **Indstillinger → Notifikationer** restart this instance,
All three together let **Settings → Notifications** restart this instance,
which is also how it picks up a new version: a container cannot replace its own
image, so the panel recreates it and pulls `:latest` on the way. Any one of the
image, so the panel recreates it and pulls `:latest` on the way. The request asks
the panel for a *scheduled* restart rather than an immediate one — an immediate
one is carried out while the request is still open, and stopping the container
kills the very process waiting for the answer, so the panel never gets to the
half that starts it again. Any one of the
three alone does nothing, and the page names the ones that are missing.
The token is a panel credential with control over that server. Anybody who can
+30
View File
@@ -22,6 +22,9 @@ forward mail to, and an MCP endpoint so Claude can work your task list directly.
Firma, labelled regnskab. Danish and English, mixed freely in one line.
- **Projects, sections, sub-tasks, labels, [saved filters](filters.md).** List,
board and calendar views, and foldable groups over the projects in the sidebar.
- **Notes, beside the work.** Rich text to write in, Markdown on disk. A note is
shared by filing it in a project; `#project` and `[[another note]]` are links,
and a task shows what has been written about it.
- **Drag where it means something** — reorder a list or a board, file a project
under a group, drop a task on another project, or drop it on another day.
- **Sharing** with owner, editor and viewer roles. Anyone outside your instance
@@ -31,6 +34,33 @@ forward mail to, and an MCP endpoint so Claude can work your task list directly.
- **[Calendar](caldav.md), [mail](mail.md) and [Claude](mcp.md)** connected over
open standards rather than private formats.
## What it looks like
![Upcoming as a month](screenshots/upcoming.png)
*Upcoming, as a month. A week and a plain list are the other two, and a task can be
dragged from one day onto another.*
![A project as a list](screenshots/sections.png)
*The same project as a list, where sections read as bands. As a board they are
columns.*
![Waiting on others](screenshots/delegated.png)
*What you have handed to somebody else, grouped by whoever has it.*
![Settings, integrations](screenshots/settings.png)
*Where [Gmail](mailboxes.md), the [calendar feed](caldav.md) and the
[mail-to-task address](mail.md) are connected.*
![Settings, people](screenshots/admin.png)
*Inviting somebody to the instance. There is no open registration — everybody
arrives through a link, and picks their own password. See
[signing in](signing-in.md).*
## Getting it running
verdande is built to be deployed as a **Rune** in
+87
View File
@@ -0,0 +1,87 @@
# Mailboxes
Star a mail and it becomes a task.
This is the other direction from [mail to task](mail.md): instead of forwarding
something, Verdande reads a mailbox you connect and makes a task out of anything
you have flagged. Nothing else in the mailbox is touched, and nothing is ever
sent.
Each mailbox belongs to the person who connected it. Two people on one instance
connect their own and see only their own — there is no shared inbox and no
administrator who can read yours.
## Gmail
**Settings → Integrations → Gmail → Connect.**
Gmail needs an OAuth client, which is the instance's registration with Google
rather than anybody's account — see [Configuration](configuration.md#google).
Once it is set up, each person connects their own mailbox through it.
Choose what makes a task:
| Trigger | What it reads |
|---|---|
| Starred | Anything you have starred |
| Label | Anything carrying a label you name |
!!! note "The app stays Internal if your client is Internal"
An OAuth client registered as *Internal* in a Google Workspace can only be
used by accounts in that workspace. A personal `@gmail.com` address will be
refused with `org_internal`, and that is the client working as configured.
Switching to *External* means refresh tokens that expire every seven days
until Google has verified the app — for one personal account, forwarding to
your [mail-to-task address](mail.md) is the shorter road.
## iCloud, Fastmail and anything else over IMAP
**Settings → Integrations → Mailboxes → Add a mailbox.**
You need three things:
| Field | Example |
|---|---|
| IMAP server | `imap.mail.me.com:993` |
| Username | your address |
| App password | not your ordinary password — see below |
The connection is tested before it is saved. A mailbox that cannot be read is
refused while you are still looking at the form, rather than accepted and then
failing quietly every ten minutes.
Flag a message — the flag, not unread — and it becomes a task on the next sweep.
Unread is a state you are still using yourself, and a sync that consumed it would
be taking something away.
### App passwords
Every host worth naming issues a separate password for applications, which you
can withdraw on its own without changing the one you sign in with. Use that.
- **iCloud** — [appleid.apple.com](https://appleid.apple.com) → Sign-In and
Security → App-Specific Passwords
- **Fastmail** — Settings → Privacy & Security → Integrations → New app password
The password is [encrypted at rest](configuration.md#secrets), and it is never
returned by the API once saved.
### Office 365 does not work here
Microsoft has closed IMAP to passwords; it wants its own OAuth flow. There is no
app password that will get you in, and a mailbox pointed at `outlook.office365.com`
will be refused. Forward to your [mail-to-task address](mail.md) instead.
## How often
Every ten minutes, in the background. **Fetch now** does it immediately, which is
for when you have just connected something and want to see it work.
A run reads at most twenty-five messages, so a mailbox with years of flagged mail
in it fills up over several sweeps rather than in one.
## Disconnecting
**Disconnect** on the mailbox. The tasks it made stay — they are your work now,
not the mailbox's. Nothing is read after that, and the stored credential is
deleted rather than merely disabled.
+736
View File
@@ -263,6 +263,36 @@ paths:
"204": { description: Saved. }
"400": { $ref: "#/components/responses/BadRequest" }
/api/v1/auth/nav-order:
put:
tags: [auth]
summary: The order of the fixed views in the sidebar.
description: |
I dag, Kommende, Venter and the inbox, in whatever order this person wants
them. On the account rather than in the browser: which order you want them
in is a fact about you and should be the same on the phone as on the laptop.
A list of keys rather than a number per view. The set of views changes with
the program, and a key that no longer exists costs nothing — the sidebar
draws what it recognises and appends anything new at the end, which is what
makes adding a view later not a migration. An empty list means the order the
program ships with.
requestBody:
required: true
content:
application/json:
schema:
type: object
properties:
order:
type: array
maxItems: 20
items: { type: string }
example: ["inbox", "today", "upcoming", "delegated"]
responses:
"204": { description: Saved. }
"400": { $ref: "#/components/responses/BadRequest" }
/api/v1/auth/me:
get:
tags: [auth]
@@ -1366,6 +1396,204 @@ paths:
description: The environment variables that are not set.
"403": { $ref: "#/components/responses/Forbidden" }
/api/v1/notes/bulk:
post:
tags: [notes]
summary: Archive, unarchive or delete a set of notes in one call.
description: |
One call rather than one per note, because the thing being asked is one
thing: "put these away". Fifty separate requests would be fifty round
trips, fifty chances for half of them to fail, and a list that redraws
fifty times — and after an import of twelve hundred notes, selecting fifty
is the ordinary case rather than the extreme one.
Each note is still checked on its own. A set is a convenience for the
caller and never a way to touch something they could not have touched one
at a time; anything they may not write to is counted in `skipped` rather
than failing the whole call, so one note somebody else just deleted does
not stop the other forty-nine.
Archiving is not deleting. The trash says "this was a mistake, and in
thirty days it is gone"; the archive says "this is finished, and I want to
keep it".
requestBody:
required: true
content:
application/json:
schema:
type: object
required: [ids]
properties:
ids:
type: array
items: { type: string }
description: At most 500.
archived:
type: boolean
description: Put away, or bring back out.
delete:
type: boolean
description: Into the trash instead. Wins over `archived`.
responses:
"200":
description: How many were changed, and how many were passed over.
content:
application/json:
schema:
type: object
properties:
done: { type: integer }
skipped: { type: integer }
"422":
description: No ids, too many, or neither archived nor delete was asked for.
/api/v1/notes/{noteID}/attachments:
post:
tags: [notes]
summary: Put a file into a note.
description: |
A picture pasted or dropped into a note. Until this existed the import
could bring files in and nothing else could, so the most ordinary act in
a notes app — paste a screenshot — did nothing at all.
Editor rights on the note. The same answer for "no such note" and "not
yours", so an id cannot be probed.
The response carries the attachment's URL; the editor writes that into the
note's text as a Markdown image. The file is never inlined as a data URL —
a photograph as base64 would make the note itself megabytes of text.
parameters:
- name: noteID
in: path
required: true
schema: { type: string }
requestBody:
required: true
content:
multipart/form-data:
schema:
type: object
required: [file]
properties:
file:
type: string
format: binary
responses:
"201":
description: Stored, and hung on the note.
"404":
description: No such note, or not yours to write in.
"413":
description: Larger than 25 MB.
/api/v1/beacon:
post:
tags: [auth]
summary: Report that an installation exists. Unauthenticated.
description: |
The receiving end of the beacon. Open on purpose: the caller is a
stranger's installation, which is the whole feature.
Answers 404 unless this instance has been told it is the collector, so an
ordinary installation is not a spam sink that answers 200 to anything.
Rate limited like the login routes, and the table behind it has a ceiling:
the endpoint inserts a row keyed on an id the caller invented, so without
one anybody could grow it until the volume is full.
Nothing about the request is recorded but the two values in the body. No
source address is read or written here.
requestBody:
required: true
content:
application/json:
schema:
type: object
required: [instance_id]
properties:
instance_id:
type: string
description: A random id the installation generated for itself.
version:
type: string
responses:
"204":
description: Recorded.
"400":
description: The id or the version is not the expected shape.
"404":
description: This instance is not a collector.
/api/v1/beacon/count:
get:
tags: [auth]
summary: How many installations reported in the last thirty days.
description: |
Unauthenticated and cached for a minute, and only answered when the
collector has been told to publish it.
A window rather than a running total: a total only climbs, so it stops
meaning "how many people use this" and starts meaning "how many ever
tried it".
responses:
"200":
description: The count.
content:
application/json:
schema:
type: object
properties:
installs: { type: integer }
window_days: { type: integer }
"404":
description: Not a collector, or not published.
/api/v1/beacon/settings:
get:
tags: [auth]
summary: What this instance reports, and to whom. Administrators only.
description: |
Includes the exact payload that will be sent, so the settings page can
print it in full rather than describe it.
responses:
"200":
description: The current configuration, and the counts if this is the collector.
content:
application/json:
schema:
type: object
properties:
enabled: { type: boolean }
collector_url: { type: string }
instance_id: { type: string }
version: { type: string }
is_collector: { type: boolean }
publish_count: { type: boolean }
last_ping_at: { type: string, format: date-time }
put:
tags: [auth]
summary: Turn the beacon off, or point it somewhere else. Administrators only.
requestBody:
required: true
content:
application/json:
schema:
type: object
properties:
enabled: { type: boolean }
collector_url:
type: string
description: |
Where reports go. Fetched through a dialer that refuses
loopback and private address space, so this cannot be used to
make the server reach inside its own network.
is_collector: { type: boolean }
publish_count: { type: boolean }
responses:
"200":
description: The configuration as it now stands.
"422":
description: The collector address is not an http or https URL.
/api/v1/panel/restart:
post:
tags: [auth]
@@ -2477,6 +2705,363 @@ paths:
created: { type: integer }
"502": { $ref: "#/components/responses/Upstream" }
/api/v1/calendar:
get:
tags: [integrations]
summary: The Google Calendar connection and the calendars in it.
description: |
Read-only. verdande shows a Google calendar beside its own dated tasks and
never writes one back — writing needs the `calendar.events` scope and an
answer to what happens when both sides moved the same meeting, which is a
synchronisation model rather than a scope.
The list of calendars is fetched from Google when it is empty, so an
account that connected during a hiccup corrects itself on the next look
rather than showing an empty panel for ever.
responses:
"200": { $ref: "#/components/responses/Calendar" }
delete:
tags: [integrations]
summary: Disconnect the calendar and forget its tokens.
description: >
The cached events go too. Unlike the tasks a mailbox made, none of this is
the person's own work — it is a copy of somebody else's calendar, and it
means nothing without the account.
responses:
"204": { description: Disconnected. }
/api/v1/calendar/calendars:
put:
tags: [integrations]
summary: Choose which calendars are drawn.
description: >
The whole set in one write rather than one flip at a time, for the reason
the projects' order is written the same way: the question is "these ones",
and a list built out of several requests can land half applied. A calendar
turned off has its cached events removed; they come back if it is turned
on again.
requestBody:
required: true
content:
application/json:
schema:
type: object
properties:
shown:
type: array
items: { type: string }
responses:
"204": { description: Saved. }
"404": { $ref: "#/components/responses/NotFound" }
/api/v1/calendar/authorize:
post:
tags: [integrations]
summary: Begin the OAuth flow for a calendar.
description: |
The same registration Gmail signs in through, asking for
`calendar.readonly` and coming back to `/oauth/calendar/callback`. Google
issues a refresh token per authorisation, so connecting a calendar does not
disturb a working Gmail connection.
responses:
"200":
description: OK
content:
application/json:
schema:
type: object
properties:
url: { type: string }
"409":
$ref: "#/components/responses/Conflict"
/api/v1/calendar/sync:
post:
tags: [integrations]
summary: Refresh the chosen calendars now.
description: >
Refreshes the list of calendars and replaces the cached window of events
for the ones that are shown. `partial` means the budget ran out before the
last calendar; what was read is kept, and the rest arrive on the next run.
responses:
"200":
description: OK
content:
application/json:
schema:
type: object
properties:
events: { type: integer }
partial: { type: boolean }
"502": { $ref: "#/components/responses/Upstream" }
/api/v1/calendar/events:
get:
tags: [integrations]
summary: The events of the chosen calendars, between two days.
description: |
Both ends inclusive, and an event that merely overlaps the range comes
back — a grid asks which cells an event covers, not where it begins.
`from` and `to` in the answer are the window verdande holds a copy of,
which is not the window that was asked for. A grid paged past it gets no
events, and an empty answer with no explanation is a calendar quietly
claiming a day is clear.
parameters:
- { name: from, in: query, required: true, schema: { type: string, format: date } }
- { name: to, in: query, required: true, schema: { type: string, format: date } }
responses:
"200":
description: OK
content:
application/json:
schema:
type: object
properties:
events:
type: array
items: { $ref: "#/components/schemas/CalendarEvent" }
from: { type: string, format: date }
to: { type: string, format: date }
"422": { $ref: "#/components/responses/Validation" }
/api/v1/export/notes.zip:
get:
tags: [notes]
summary: Every note as a Markdown file, in a zip.
description: >
One file per note, named after its title, containing exactly what is
stored. Nothing is converted on the way out — the note on disk is already
the file you would export — so it opens in Obsidian, in a text editor, or
in anything else that reads Markdown.
responses:
"200":
description: A zip archive.
content:
application/zip:
schema: { type: string, format: binary }
/api/v1/notes:
get:
tags: [notes]
summary: Notes — listed, narrowed to a project, or searched.
description: >
With no parameters: the notes filed under no project, which are the
person's own. With `project`: that project's, which needs read access to
it. With `q`: a full-text search across every note they can see, matching
Danish spelling either way round — "grøn" finds "gron" and the reverse.
parameters:
- name: q
in: query
schema: { type: string }
- name: project
in: query
schema: { type: string }
responses:
"200":
description: OK
content:
application/json:
schema:
type: object
properties:
notes:
type: array
items: { $ref: "#/components/schemas/Note" }
post:
tags: [notes]
summary: Write a note.
requestBody:
required: true
content:
application/json:
schema: { $ref: "#/components/schemas/NoteInput" }
responses:
"201":
description: Written
content:
application/json:
schema: { $ref: "#/components/schemas/Note" }
"403": { $ref: "#/components/responses/Forbidden" }
/api/v1/notes/import:
post:
tags: [notes]
summary: A zip of Markdown files becomes notes.
description: >
The mirror of the export, and the way in from anywhere else: a folder of
Markdown is what Obsidian, Bear, iA Writer and a shell script all produce.
Filenames are ignored — the title is the first line of the body, here as
everywhere, and taking it from the filename would be a second rule for the
same thing. Directories, macOS resource forks and anything that is not .md
are passed over.
requestBody:
required: true
content:
multipart/form-data:
schema:
type: object
properties:
file: { type: string, format: binary }
responses:
"200":
description: Read
content:
application/json:
schema:
type: object
properties:
created: { type: integer }
skipped: { type: integer, description: Files that were not usable as a note. }
"413": { description: The archive is too large. }
/api/v1/notes/{noteID}:
parameters:
- name: noteID
in: path
required: true
schema: { type: string }
get:
tags: [notes]
summary: One note.
responses:
"200":
description: OK
content:
application/json:
schema: { $ref: "#/components/schemas/Note" }
"404": { $ref: "#/components/responses/NotFound" }
patch:
tags: [notes]
summary: Change a note. Every field is optional; what is sent is what changes.
requestBody:
required: true
content:
application/json:
schema: { $ref: "#/components/schemas/NoteInput" }
responses:
"200":
description: OK
content:
application/json:
schema: { $ref: "#/components/schemas/Note" }
"404": { $ref: "#/components/responses/NotFound" }
delete:
tags: [notes]
summary: Put a note in the trash. It can be brought back.
responses:
"204": { description: Deleted }
"404": { $ref: "#/components/responses/NotFound" }
/api/v1/notes/linking/{kind}/{targetID}:
get:
tags: [notes]
summary: What has been written about this.
description: >
The backwards direction. A task's id gives every note that mentions it,
a project's name gives every note carrying that #tag.
parameters:
- name: kind
in: path
required: true
schema: { type: string, enum: [task, note, project] }
- name: targetID
in: path
required: true
schema: { type: string }
responses:
"200":
description: OK
content:
application/json:
schema:
type: object
properties:
notes:
type: array
items: { $ref: "#/components/schemas/Note" }
"400": { $ref: "#/components/responses/BadRequest" }
/api/v1/mailboxes:
get:
tags: [integrations]
summary: The mailboxes this person has connected.
responses:
"200":
description: OK
content:
application/json:
schema:
type: object
properties:
mailboxes:
type: array
items: { $ref: "#/components/schemas/Mailbox" }
post:
tags: [integrations]
summary: Connect a mailbox over IMAP.
description: >
The connection is tested before it is saved, so a mailbox that cannot be
read is refused here rather than failing quietly every ten minutes.
requestBody:
required: true
content:
application/json:
schema:
type: object
required: [host, username, password]
properties:
name: { type: string }
host: { type: string, example: "imap.mail.me.com:993" }
username: { type: string }
password: { type: string, description: "An app-specific password on every host worth naming. Stored sealed." }
folder: { type: string, default: INBOX }
responses:
"201":
description: Connected
content:
application/json:
schema: { $ref: "#/components/schemas/Mailbox" }
"400": { $ref: "#/components/responses/BadRequest" }
"409": { description: That mailbox is already connected }
"422": { $ref: "#/components/responses/Upstream" }
/api/v1/mailboxes/{mailboxID}:
delete:
tags: [integrations]
summary: Disconnect a mailbox. The tasks it made stay.
parameters:
- name: mailboxID
in: path
required: true
schema: { type: string }
responses:
"204": { description: Disconnected }
/api/v1/mailboxes/{mailboxID}/sync:
post:
tags: [integrations]
summary: Read one mailbox now.
parameters:
- name: mailboxID
in: path
required: true
schema: { type: string }
responses:
"200":
description: OK
content:
application/json:
schema:
type: object
properties:
created: { type: integer }
partial:
type: boolean
description: The run hit its time budget. What it made is kept.
"404": { $ref: "#/components/responses/NotFound" }
"422": { $ref: "#/components/responses/Upstream" }
/api/v1/ai/settings:
get:
tags: [integrations]
@@ -2883,6 +3468,26 @@ paths:
responses:
"302": { description: "Back into the app, with the outcome as a query parameter." }
/oauth/calendar/callback:
get:
tags: [integrations]
summary: Where Google sends the browser after consent to a calendar.
description: |
A URI of its own rather than a reuse of Gmail's. One callback deciding from
stored state which of the two flows had come back would put the choice of
which token store to write behind a lookup — and when that goes wrong it
goes wrong by overwriting a working Gmail connection with calendar tokens.
Answers with a redirect to `/indstillinger/integrationer`, not with JSON.
An `org_internal` error here means the OAuth app is registered as Internal
and the account is outside the organisation.
parameters:
- { name: code, in: query, schema: { type: string } }
- { name: state, in: query, schema: { type: string } }
- { name: error, in: query, schema: { type: string } }
responses:
"302": { description: "Back into the app, with the outcome as a query parameter." }
/mcp:
post:
tags: [integrations]
@@ -3065,6 +3670,40 @@ components:
email: { type: string }
trigger: { type: string, enum: ["", starred, label, both] }
label: { type: string }
Calendar:
description: OK
content:
application/json:
schema:
type: object
properties:
connected: { type: boolean }
account:
type: string
description: >
The address of the connected account, taken from the id of its
primary calendar — which is where Google puts it, so no profile
scope has to be asked for to learn it.
read_only:
type: boolean
description: >
Always true today. Stated rather than assumed: an interface that
quietly looks editable is one somebody will try to edit.
has_client:
type: boolean
description: >
Whether an OAuth client is registered on this instance at all, so
the page can explain rather than offer a button that answers 409.
redirect_uri:
type: string
description: >
What has to be registered with Google, spelled out. Derived from
VERDANDE_BASE_URL and matched exactly — the single most likely
thing to be wrong, and the error Google gives names neither value.
last_sync_at: { type: string, format: date-time }
calendars:
type: array
items: { $ref: "#/components/schemas/Calendar" }
ImportResult:
description: Created
content:
@@ -3082,6 +3721,103 @@ components:
items: { type: string }
schemas:
Calendar:
type: object
description: >
One calendar inside a connected account. Kept whether or not it is shown,
so turning one off and on again gives back the same row.
properties:
id: { type: string }
remote_id: { type: string, description: "The id Google knows it by; an address for the primary calendar." }
name: { type: string }
colour:
type: string
description: >
Google's own hex value, not one of verdande's ten named colours. Those
are names so the interface can restyle them per theme and they are
verdande's palette; a Google calendar is told apart by the colour the
person already knows it by.
time_zone: { type: string }
primary: { type: boolean }
writable:
type: boolean
description: >
Whether the connected account could write to it. Nothing does — the
scope is read-only — and it is stored so the day something can, the
interface knows which calendars could even be offered.
shown: { type: boolean }
CalendarEvent:
type: object
description: >
One occurrence, in the shape a grid asks about. A recurrence is expanded by
Google into its occurrences, so what arrives here is a day, not a rule.
properties:
id: { type: string }
calendar_id: { type: string }
summary: { type: string }
starts_at:
type: string
description: >
RFC 3339 as Google wrote it, offset and all. Empty for an all-day
event, which has a day rather than a moment — inventing midnight for it
is how an event moves a day when a container moves time zone.
ends_at: { type: string }
start_day: { type: string, format: date }
end_day:
type: string
format: date
description: >
The last day the event covers, inclusive. Google's all-day end date is
exclusive and is corrected on the way in, once, rather than by every
reader.
all_day: { type: boolean }
location: { type: string }
url: { type: string, description: "Opens the event in Google's own interface." }
calendar_name: { type: string }
colour: { type: string }
Note:
type: object
description: A page of Markdown. The body is the stored form and the exported form.
properties:
id: { type: string }
project_id: { type: string }
title: { type: string, readOnly: true, description: "The note's first line. Derived on every save; it cannot be set." }
body: { type: string, description: Markdown }
pinned: { type: boolean }
created_at: { type: string, format: date-time }
updated_at: { type: string, format: date-time }
links:
type: array
description: What the body points at, read out of the text on the way in.
items:
type: object
properties:
kind: { type: string, enum: [task, note, project] }
target_id: { type: string }
NoteInput:
type: object
properties:
project_id: { type: string, description: "Empty files it under no project." }
body: { type: string, description: "Markdown. Its first line becomes the title." }
pinned: { type: boolean }
Mailbox:
type: object
description: One connected mailbox. The password and any token are never returned.
properties:
id: { type: string }
kind: { type: string, enum: [gmail, imap] }
name: { type: string }
host: { type: string }
username: { type: string }
folder: { type: string }
last_uid: { type: integer }
last_sync_at: { type: string, format: date-time }
created_at: { type: string, format: date-time }
Error:
type: object
description: |
+18 -3
View File
@@ -70,7 +70,22 @@ an error message and a lost thought.
write it, and guessing month-first would silently move a date by months rather
than failing visibly.
## The shortcuts
## Just start typing
Press ++q++ to jump to the quick-add box from anywhere, ++t++ for Today, ++u++ for
Upcoming, ++cmd+k++ to search, and ++question++ for the full list.
There is no shortcut to remember. Press a letter anywhere in the app and you are
writing a task, with the letter you pressed already in the field. The thought does
not have to survive a journey to find somewhere to put it.
It only fires when nothing else has focus — a key pressed at a field, a button or
a link belongs to that thing.
Under the field is a line saying what the parser reads: `#` project, `/` section,
`@` label, `p1` priority, and an example date. It appears while you are typing,
which is the one moment a placeholder cannot help you.
++cmd+k++ opens the palette — search, and navigation to anywhere in the app.
!!! note "The single letters used to navigate"
++t++ for Today and ++u++ for Upcoming are gone. One keyboard cannot both
spell *tal med Anders* and jump to Today on the same ++t++, and capturing is
the thing done fifty times a day. The palette does the rest.
Binary file not shown.

Before

Width:  |  Height:  |  Size: 166 KiB

After

Width:  |  Height:  |  Size: 239 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 181 KiB

After

Width:  |  Height:  |  Size: 276 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 91 KiB

After

Width:  |  Height:  |  Size: 136 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 228 KiB

After

Width:  |  Height:  |  Size: 306 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 243 KiB

After

Width:  |  Height:  |  Size: 241 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 328 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 308 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 176 KiB

After

Width:  |  Height:  |  Size: 215 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 230 KiB

After

Width:  |  Height:  |  Size: 240 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 199 KiB

After

Width:  |  Height:  |  Size: 232 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 154 KiB

After

Width:  |  Height:  |  Size: 222 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 153 KiB

After

Width:  |  Height:  |  Size: 221 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 188 KiB

After

Width:  |  Height:  |  Size: 209 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 167 KiB

After

Width:  |  Height:  |  Size: 180 KiB

+39
View File
@@ -0,0 +1,39 @@
# Signing in
Three ways in, and you can have all of them at once.
## A password
What the first account is created with. Nothing is emailed, nothing is registered
anywhere: the first person to open a fresh instance creates the administrator
account, and after that people arrive by invite link.
## A code on top of it
**Settings → Account → Two-factor.** An ordinary TOTP secret — any authenticator
app reads the QR code. Once it is on, the password alone is not enough.
Recovery codes are shown once, when you turn it on. They are the way back if the
phone is gone, and they are not shown again.
## A passkey
**Settings → Account → Passkeys → Add a passkey.**
A key on a device instead of a secret in your head. The private half never leaves
the device — a phone's secure element, a laptop's keychain, a hardware key — and
the signature is bound to the address it was made for, so a convincing copy of the
sign-in page at a different address gets nothing.
Give each one a name you will recognise later. *min bærbare* is what makes a list
of keys reviewable, and a list nobody can read is a list nobody revokes from.
Passkeys need the address in `VERDANDE_BASE_URL` to be a hostname. Browsers refuse
to make one for a bare IP address, so an instance reached at `http://192.168.1.10:8080`
will show the passkey section as unavailable rather than failing at the moment
somebody tries.
!!! note "One is enough, two is better"
Lose the only device holding your only passkey and you are back to the
password. Keep the password, or register a second key — a phone as well as a
laptop.
+37 -2
View File
@@ -19,6 +19,8 @@ import (
"fmt"
"io"
"net/http"
"github.com/kristianwind/verdande/internal/safedial"
"strings"
"time"
)
@@ -76,7 +78,16 @@ func New(cfg Config) *Client {
// Generous: a local model on modest hardware can take a while to produce
// its first token, and the alternative is a timeout that only ever fires
// for the people running their own.
http: &http.Client{Timeout: 120 * time.Second},
// safedial, not a plain client. `base_url` is typed into a field in the
// interface, so it is a user-supplied address — and a server that fetches a
// user-supplied address is a way to ask it to reach things the caller
// cannot: the panel next door, a database on the same bridge, the cloud
// metadata endpoint. On a homelab that is most of what is worth reaching.
//
// The check is on the resolved address rather than on the URL, because a
// name answers whatever its owner says — and can answer differently the
// second time, after a parse-time check has passed.
http: safedial.Client(120 * time.Second),
}
}
@@ -268,11 +279,35 @@ func (c *Client) do(req *http.Request, out any) error {
// The provider's own message is included: for a local model it is usually
// the actual problem ("model not found"), and hiding it would leave the
// operator with nothing to go on.
return fmt.Errorf("ai: %s said %s: %s", req.URL.Host, resp.Status, truncate(string(raw), 300))
// The provider's body is no longer passed back to the caller.
//
// It was, and combined with a free-text base URL that made a working
// internal scanner: point it at a host, read the first three hundred bytes
// of whatever answered. The operator still needs the message — for a local
// model it is usually the actual problem — so it goes to the log, where it
// is available to the person running the instance and to nobody else.
return &upstreamError{Host: req.URL.Host, Status: resp.Status, Body: truncate(string(raw), 300)}
}
return json.Unmarshal(raw, out)
}
// upstreamError keeps the provider's own words for the log and away from the
// response. Error() is what reaches a caller; Detail() is what is written down.
type upstreamError struct {
Host string
Status string
Body string
}
func (e *upstreamError) Error() string {
return fmt.Sprintf("ai: %s said %s", e.Host, e.Status)
}
// Detail is the whole of it, for the instance's own error log.
func (e *upstreamError) Detail() string {
return fmt.Sprintf("ai: %s said %s: %s", e.Host, e.Status, e.Body)
}
func truncate(s string, max int) string {
if len(s) <= max {
return s
+139
View File
@@ -0,0 +1,139 @@
// Package beacon answers one question for the project, and refuses to answer any
// other: how many installations are out there.
//
// It sends two values once a day — an anonymous id the installation made up for
// itself, and the version it is running — and nothing else. No IP is stored by
// the collector, no hostname, no domain, no account, no usage. The payload is
// small enough to be printed in full on the settings page, and it is, because a
// promise about telemetry is worth exactly as much as the reader's ability to
// check it.
//
// # On by default, and off in one click
//
// The alternative that was asked for first was mandatory, and the argument
// against it is practical rather than principled: it cannot be enforced. The
// domain can be blocked, the binary can be patched, and a fork removes it in its
// first commit — so mandatory buys nothing over on-by-default, while costing the
// trust the rest of this program works for. Secrets are sealed here, the key is
// kept out of the database, and the backup page warns you what a copy contains.
// Telemetry you are not allowed to refuse is the one sentence in that story that
// does not fit.
//
// So: on by default, said plainly, one switch to turn it off. That collects
// essentially the same number and leaves nobody feeling tricked.
//
// # Where this disagrees with the rest of the program
//
// `config.UpdateCheck` is off unless asked for, and its comment gives the reason:
// "a self-hosted app that reaches out without being told to has broken the deal
// its operator made by self-hosting." That is the same situation as this one, and
// the two defaults do not agree.
//
// The disagreement is written down here rather than smoothed over, because it is
// a real one and the reader deserves to see both halves. The case for treating
// them differently is that the update check reveals an instance to a third party
// (GitHub) and returns something the operator can act on, while this returns
// nothing and goes to the project itself — and that a count nobody sends is a
// count that does not exist, whereas an update nobody checks for is one you can
// still go and look up. Whether that is a distinction or an excuse is a judgement
// call, and it was made deliberately by the person whose instance this is.
//
// If it is ever decided the other way, the change is one word: the default in
// `beaconSettings`. Nothing else in this package assumes which way it points.
//
// # What the collector can still see
//
// A request has a source address whether or not anybody writes it down. The
// collector does not store or log it, and the code is arranged so that it never
// reaches a function that could — but it passes through a reverse proxy and a
// tunnel on the way, and those keep their own logs. Saying "we send no IP" would
// be a half-truth; what is true is that nothing here records one.
package beacon
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"regexp"
"time"
)
// DefaultCollector is where a fresh installation reports, unless it is told
// otherwise. Somebody running their own fleet points it at their own instance;
// the field is on the settings page for exactly that.
const DefaultCollector = "https://verdande.yggdrasilpanel.com/api/v1/beacon"
// Payload is the entire message. Two fields, and adding a third is a change to
// what this program promises, not a change to a struct.
type Payload struct {
InstanceID string `json:"instance_id"`
Version string `json:"version"`
}
// InstanceID and Version are checked on the way in as well as written on the way
// out, because the collector's endpoint is unauthenticated: anything that reaches
// it is a stranger's bytes, and it should be able to store only the two shapes it
// expects.
var (
idShape = regexp.MustCompile(`^[0-9a-fA-F-]{16,64}$`)
versionShape = regexp.MustCompile(`^v?[0-9]{1,4}(\.[0-9]{1,5}){0,3}(-[0-9A-Za-z.]{1,20})?$`)
)
// ValidID reports whether an id is the shape this program writes.
func ValidID(s string) bool { return idShape.MatchString(s) }
// ValidVersion allows a version number and the word this binary uses when it was
// built outside a release.
func ValidVersion(s string) bool { return s == "dev" || versionShape.MatchString(s) }
// Send delivers one ping. It is deliberately dull: a short timeout, no retry, no
// queue, and a failure that is returned rather than made anybody's problem.
//
// No retry, because the next pass is tomorrow and a missed day changes a count of
// installations by nothing. A telemetry ping that retries is a telemetry ping
// that can hammer somebody's server on the day it is down.
func Send(ctx context.Context, client *http.Client, collector string, p Payload) error {
if !ValidID(p.InstanceID) {
return fmt.Errorf("beacon: refusing to send a malformed id")
}
// Versionen kontrolleres også her, ikke kun hos modtageren. Kommentaren ved
// mønstrene lovede begge veje, og gjorde det ikke — og en installation, der
// sender noget mærkeligt, skal opdage det hos sig selv frem for at være noget,
// en fremmed collector skal filtrere fra.
if !ValidVersion(p.Version) {
return fmt.Errorf("beacon: refusing to send a malformed version")
}
body, err := json.Marshal(p)
if err != nil {
return err
}
ctx, cancel := context.WithTimeout(ctx, 15*time.Second)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodPost, collector, bytes.NewReader(body))
if err != nil {
return err
}
req.Header.Set("Content-Type", "application/json")
// A plain user agent with the version already in the payload. Nothing is
// learned from it that the body does not already say, and it makes the request
// legible in a log rather than anonymous-looking.
req.Header.Set("User-Agent", "verdande-beacon/1")
resp, err := client.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
// Read and discard, so the connection can be reused rather than dropped.
_, _ = io.Copy(io.Discard, io.LimitReader(resp.Body, 4<<10))
if resp.StatusCode >= 400 {
return fmt.Errorf("beacon: collector answered %d", resp.StatusCode)
}
return nil
}
+114
View File
@@ -0,0 +1,114 @@
package beacon
import (
"context"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"testing"
)
// Hvad der bliver sendt, tegn for tegn.
//
// Prøven er skrevet imod *hele* kroppen og ikke imod enkelte felter, fordi det er
// den, der er løftet: siden siger "præcis det her og ikke mere", og et felt lagt
// til senere skal vælte noget. En prøve, der kun så efter, at instance_id var med,
// ville lade en ny nøgle glide igennem uden at nogen opdagede det.
func TestSendsExactlyTwoFields(t *testing.T) {
var got map[string]any
var contentType string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
contentType = r.Header.Get("Content-Type")
body, _ := io.ReadAll(r.Body)
if err := json.Unmarshal(body, &got); err != nil {
t.Errorf("kroppen er ikke JSON: %v", err)
}
w.WriteHeader(http.StatusNoContent)
}))
defer srv.Close()
err := Send(context.Background(), srv.Client(), srv.URL, Payload{
InstanceID: "01a01b16-4e54-7000-9eaa-2f8eb0629c3a",
Version: "v0.26.3",
})
if err != nil {
t.Fatalf("Send: %v", err)
}
if contentType != "application/json" {
t.Errorf("Content-Type = %q", contentType)
}
if len(got) != 2 {
t.Fatalf("der blev sendt %d felter, ikke to: %v", len(got), got)
}
if got["instance_id"] != "01a01b16-4e54-7000-9eaa-2f8eb0629c3a" {
t.Errorf("instance_id = %v", got["instance_id"])
}
if got["version"] != "v0.26.3" {
t.Errorf("version = %v", got["version"])
}
}
// En collector, der er nede, er ikke denne installations problem.
func TestSendReportsAFailureRatherThanRetrying(t *testing.T) {
calls := 0
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
calls++
w.WriteHeader(http.StatusInternalServerError)
}))
defer srv.Close()
err := Send(context.Background(), srv.Client(), srv.URL, Payload{
InstanceID: "01a01b16-4e54-7000-9eaa-2f8eb0629c3a", Version: "v1.0.0",
})
if err == nil {
t.Fatal("en 500 skal give en fejl")
}
if calls != 1 {
t.Errorf("der blev prøvet %d gange; en telemetri-ping må ikke hamre på en server, der er nede", calls)
}
}
// Et id, der ikke er id-formet, sendes slet ikke — så en fejl her ikke bliver til
// noget, en fremmed collector skal filtrere fra.
func TestSendRefusesAMalformedID(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
t.Error("der blev sendt noget, som ikke skulle sendes")
}))
defer srv.Close()
if err := Send(context.Background(), srv.Client(), srv.URL, Payload{
InstanceID: "; DROP TABLE beacon_installs", Version: "v1.0.0",
}); err == nil {
t.Fatal("et misformet id skal afvises før det forlader maskinen")
}
}
func TestShapes(t *testing.T) {
ok := []string{"01a01b16-4e54-7000-9eaa-2f8eb0629c3a", "89df44d7-3742-4611-9a43-db0b07733c6d"}
for _, s := range ok {
if !ValidID(s) {
t.Errorf("ValidID(%q) = false", s)
}
}
bad := []string{"", "kort", "../../etc/passwd", "01a01b16-4e54-7000-9eaa-2f8eb0629c3a'; --",
"<script>alert(1)</script>"}
for _, s := range bad {
if ValidID(s) {
t.Errorf("ValidID(%q) = true", s)
}
}
for _, s := range []string{"v0.26.3", "0.26.3", "v1.2.3-rc.1", "dev"} {
if !ValidVersion(s) {
t.Errorf("ValidVersion(%q) = false", s)
}
}
for _, s := range []string{"", "v<script>", "v" + string(make([]byte, 200))} {
if ValidVersion(s) {
t.Errorf("ValidVersion(%q) = true", s)
}
}
}
+31 -12
View File
@@ -59,10 +59,19 @@ type Config struct {
// without spending the real one on every run.
GmailSyncBudget time.Duration
// GmailEndpoints redirects the Gmail calls at a server the tests control. Empty
// in production, where it means Google.
GmailTokenURL string
GmailAPIURL string
// CalendarSyncBudget is the same bound for one calendar refresh, and exists for
// the same reason: a slow Google held the request open past what Cloudflare is
// willing to wait, and the browser got Cloudflare's HTML instead of an error
// this server could explain.
CalendarSyncBudget time.Duration
// Where Google is. Empty in production; a test points them at a server it
// controls, which is the only way to exercise a Google that is slow or refuses.
// The token endpoint is shared — Gmail and Calendar sign in through the same
// registration — and the two APIs are not.
GoogleTokenURL string
GmailAPIURL string
CalendarAPIURL string
PanelURL string
PanelToken string
@@ -98,14 +107,15 @@ func Load() (*Config, error) {
TrashRetention: 30 * 24 * time.Hour,
Dev: envBool("VERDANDE_DEV", false),
GmailClientID: env("VERDANDE_GMAIL_CLIENT_ID", ""),
GmailSyncBudget: 25 * time.Second,
PanelURL: strings.TrimSuffix(env("VERDANDE_PANEL_URL", ""), "/"),
PanelToken: env("VERDANDE_PANEL_TOKEN", ""),
PanelServerID: env("VERDANDE_PANEL_SERVER_ID", ""),
GmailClientSecret: env("VERDANDE_GMAIL_CLIENT_SECRET", ""),
SecretKey: env("VERDANDE_SECRET_KEY", ""),
UpdateCheck: envBool("VERDANDE_UPDATE_CHECK", false),
GmailClientID: env("VERDANDE_GMAIL_CLIENT_ID", ""),
GmailSyncBudget: 25 * time.Second,
CalendarSyncBudget: 25 * time.Second,
PanelURL: strings.TrimSuffix(env("VERDANDE_PANEL_URL", ""), "/"),
PanelToken: env("VERDANDE_PANEL_TOKEN", ""),
PanelServerID: env("VERDANDE_PANEL_SERVER_ID", ""),
GmailClientSecret: env("VERDANDE_GMAIL_CLIENT_SECRET", ""),
SecretKey: env("VERDANDE_SECRET_KEY", ""),
UpdateCheck: envBool("VERDANDE_UPDATE_CHECK", false),
}
var err error
@@ -152,6 +162,15 @@ func (c *Config) BackupsDir() string { return filepath.Join(c.DataDir, "backups"
// configured separately and left to drift out of step with it.
func (c *Config) GmailRedirectURL() string { return c.BaseURL + "/oauth/gmail/callback" }
// CalendarRedirectURL is a second registered URI rather than a reuse of Gmail's.
//
// One callback that decided from server-side state which of the two flows had come
// back would save the operator one line in Google Cloud, and would put the choice
// of *which token store to write* behind a lookup. When that goes wrong it goes
// wrong by overwriting a working Gmail connection with calendar tokens — a failure
// nobody would think to look for. Two paths cannot do that.
func (c *Config) CalendarRedirectURL() string { return c.BaseURL + "/oauth/calendar/callback" }
func env(key, def string) string {
if v, ok := os.LookupEnv(key); ok && v != "" {
return v
+30
View File
@@ -60,6 +60,20 @@ func Parse(query string, ctx Context) (Compiled, error) {
if len(p.tokens) == 0 {
return Compiled{}, fmt.Errorf("filteret er tomt")
}
// Refused before it is parsed, not while.
//
// `combine` rebuilds the whole accumulated SQL string and re-copies the
// argument slice per operator, so the cost is quadratic in the number of
// terms: fifty thousand of them — three hundred kilobytes, which fits in a
// query string — took seven seconds of CPU and forty-six gigabytes of
// allocation in one request. The preview endpoint runs this on whatever is
// typed, and there is no write timeout on the server to cut it off.
//
// Two hundred terms is a filter nobody has ever written and far below where it
// costs anything.
if len(p.tokens) > maxTokens {
return Compiled{}, fmt.Errorf("filteret er for langt")
}
node, err := p.parseExpression()
if err != nil {
@@ -140,10 +154,19 @@ func isOperator(r rune) bool {
// --- parser -------------------------------------------------------------------
// The ceilings on one filter. See Parse for why.
const (
maxTokens = 200
maxDepth = 32
)
type parser struct {
tokens []token
pos int
ctx Context
// depth guards against a filter that is only parentheses. Each level is a
// stack frame, and nothing else stops somebody sending a megabyte of them.
depth int
}
func (p *parser) done() bool { return p.pos >= len(p.tokens) }
@@ -187,6 +210,13 @@ func (p *parser) parseTerm() (Compiled, error) {
if p.done() {
return Compiled{}, fmt.Errorf("filteret slutter midt i et udtryk")
}
// Nesting is stack depth. A filter made of half a million opening parentheses
// is a megabyte of text and a stack frame for each.
p.depth++
defer func() { p.depth-- }()
if p.depth > maxDepth {
return Compiled{}, fmt.Errorf("filteret har for mange niveauer")
}
switch t := p.peek(); t.kind {
case tokNot:
+339
View File
@@ -0,0 +1,339 @@
// Package gcal reads a Google Calendar. It never writes one.
//
// The scope is calendar.readonly, and that is the whole of the decision: verdande
// already is a CalDAV server, so a Google event shown here is a second calendar
// laid over the first, not a copy Google and verdande now both have to agree
// about. Writing needs `calendar.events`, and it needs an answer to what happens
// when both sides moved the same meeting — which is a synchronisation model, not a
// scope. Read first; write later and to one calendar at a time.
//
// What the API gives back and what this returns are deliberately different shapes.
// Google answers with RFC 3339 instants for a timed event and bare dates for an
// all-day one, and with an *exclusive* end date for the second. A calendar grid
// asks one question — "which cells does this event cover" — so that is what an
// Event answers, in days, worked out once here rather than in every caller.
package gcal
import (
"context"
"fmt"
"net/http"
"net/url"
"strings"
"time"
"github.com/kristianwind/verdande/internal/google"
)
// Scope is read-only, and only the calendar.
//
// Google also offers calendar.calendarlist.readonly and calendar.events.readonly,
// which together are strictly less than this — they leave out access rules and
// account settings, which verdande has no use for. They are the tighter pair and
// worth moving to; they are also newer, and this one has been documented and
// accepted unchanged for a decade. A consent screen that fails with `invalid_scope`
// is not a place to be adventurous on somebody else's Google account.
const Scope = "https://www.googleapis.com/auth/calendar.readonly"
const defaultAPIBase = "https://www.googleapis.com/calendar/v3"
// The OAuth flow, which is Google's rather than the calendar's — the same one
// Gmail signs in through, asking for the scope above. Aliased so a caller reads
// one package for one feature.
type (
Config = google.Config
PKCE = google.PKCE
Token = google.Token
)
var NewPKCE = google.NewPKCE
// ErrUnauthorized means the tokens are no longer good — access was revoked, or the
// password changed. Re-exported so a caller need not know the flow lives elsewhere.
var ErrUnauthorized = google.ErrUnauthorized
type Client struct {
accessToken string
// base is where the Calendar API is. A field rather than a constant so a test
// can point it at a server it controls; empty is Google. There is no other way
// to exercise a calendar that is slow, empty, or refuses.
base string
http *http.Client
}
func NewClient(accessToken string) *Client {
return &Client{accessToken: accessToken, http: &http.Client{Timeout: 30 * time.Second}}
}
// At points the client at another server. Tests only; the zero value is Google.
func (c *Client) At(base string) *Client {
c.base = base
return c
}
func (c *Client) api() string {
if c.base != "" {
return c.base
}
return defaultAPIBase
}
// Calendar is one entry in the person's calendar list.
type Calendar struct {
ID string
// Name is the summary Google holds. A calendar somebody renamed for themselves
// carries summaryOverride instead, and that is the name they will look for.
Name string
// Colour is Google's own hex value for the calendar, which is what makes two
// calendars in one grid tellable apart. Their own colour rather than one picked
// here: it is the colour they already know the calendar by.
Colour string
TimeZone string
Primary bool
// Owner means the connected account can write to it. Nothing writes today —
// it is stored so the day something does, the interface knows which calendars
// could even be offered.
Owner bool
}
// Calendars lists what the connected account can see.
func (c *Client) Calendars(ctx context.Context) ([]Calendar, error) {
var out []Calendar
page := ""
// Bounded rather than "until Google stops": a page token that never empties is
// a loop that never ends, and the difference between a bug at Google and a
// hung sync should not be this server hanging.
for range 10 {
v := url.Values{}
v.Set("minAccessRole", "reader")
v.Set("maxResults", "250")
if page != "" {
v.Set("pageToken", page)
}
var parsed struct {
Items []struct {
ID string `json:"id"`
Summary string `json:"summary"`
SummaryOverride string `json:"summaryOverride"`
BackgroundColor string `json:"backgroundColor"`
TimeZone string `json:"timeZone"`
Primary bool `json:"primary"`
AccessRole string `json:"accessRole"`
Deleted bool `json:"deleted"`
} `json:"items"`
NextPageToken string `json:"nextPageToken"`
}
if err := c.get(ctx, c.api()+"/users/me/calendarList?"+v.Encode(), &parsed); err != nil {
return nil, err
}
for _, item := range parsed.Items {
if item.Deleted {
continue
}
name := item.SummaryOverride
if name == "" {
name = item.Summary
}
out = append(out, Calendar{
ID: item.ID, Name: name, Colour: item.BackgroundColor,
TimeZone: item.TimeZone, Primary: item.Primary,
Owner: item.AccessRole == "owner" || item.AccessRole == "writer",
})
}
if parsed.NextPageToken == "" {
break
}
page = parsed.NextPageToken
}
return out, nil
}
// Event is one occurrence, in the shape a calendar grid asks about.
type Event struct {
ID string
Summary string
// StartsAt and EndsAt are RFC 3339 for a timed event and empty for an all-day
// one — an all-day event has no instant, only a day, and inventing midnight for
// it is how an event moves a day every time a server changes time zone.
StartsAt string
EndsAt string
// StartDay and EndDay are the first and last day the event covers, inclusive at
// both ends. Google's all-day end date is exclusive; that is corrected here,
// once, rather than in every reader.
StartDay string
EndDay string
AllDay bool
Location string
// URL opens the event in Google's own interface. Same reasoning as the Gmail
// link on a task made from a mail: what verdande shows is a pointer to the
// event, not a second copy of it that somebody could try to edit.
URL string
}
// Events lists what falls inside a window, one calendar at a time.
//
// `singleEvents` expands a recurrence into its occurrences, which is the only form
// a grid can draw: an RRULE is an instruction, and a Tuesday cell needs the Tuesday
// it produced. It also means verdande does not have to grow a second recurrence
// engine that agrees with Google's about Danish summer time.
//
// from and to are dates, inclusive. The window is the caller's business; this
// turns it into the half-open interval Google wants.
func (c *Client) Events(ctx context.Context, calendarID, from, to string) ([]Event, error) {
start, err := time.Parse(dateLayout, from)
if err != nil {
return nil, fmt.Errorf("gcal: from: %w", err)
}
end, err := time.Parse(dateLayout, to)
if err != nil {
return nil, fmt.Errorf("gcal: to: %w", err)
}
var out []Event
page := ""
for range 20 {
v := url.Values{}
v.Set("singleEvents", "true")
v.Set("orderBy", "startTime")
v.Set("maxResults", "2500")
// UTC, and a day of slack at each end. An event at 23:30 on the last day in
// Copenhagen is 21:30 UTC that day, but one at 00:30 on the first day is
// 22:30 UTC the day *before* — so a window cut exactly at the requested
// dates loses the events at both edges. The extra day is thrown away again
// by the day fields below, which are computed from the event's own offset.
v.Set("timeMin", start.AddDate(0, 0, -1).Format(time.RFC3339))
v.Set("timeMax", end.AddDate(0, 0, 2).Format(time.RFC3339))
if page != "" {
v.Set("pageToken", page)
}
var parsed struct {
Items []rawEvent `json:"items"`
NextPageToken string `json:"nextPageToken"`
}
url := c.api() + "/calendars/" + url.PathEscape(calendarID) + "/events?" + v.Encode()
if err := c.get(ctx, url, &parsed); err != nil {
return nil, err
}
for _, item := range parsed.Items {
// A cancelled occurrence of a recurring event still comes back, so that
// a client keeping its own copy knows to drop it. verdande replaces the
// window wholesale, so it only has to not add it.
if item.Status == "cancelled" {
continue
}
event, ok := item.event()
if !ok {
continue
}
if event.EndDay < from || event.StartDay > to {
continue // the slack above, thrown away
}
out = append(out, event)
}
if parsed.NextPageToken == "" {
break
}
page = parsed.NextPageToken
}
return out, nil
}
const dateLayout = "2006-01-02"
type rawEvent struct {
ID string `json:"id"`
Status string `json:"status"`
Summary string `json:"summary"`
Location string `json:"location"`
HTMLLink string `json:"htmlLink"`
Start when `json:"start"`
End when `json:"end"`
}
type when struct {
Date string `json:"date"`
DateTime string `json:"dateTime"`
}
// event turns Google's two shapes into the one a grid can read.
//
// The second return is false for an event with neither a date nor a dateTime —
// which Google does return, for an event somebody is still being asked about. It
// belongs on no day, so there is no cell to draw it in.
func (r rawEvent) event() (Event, bool) {
e := Event{
ID: r.ID, Summary: strings.TrimSpace(r.Summary),
Location: r.Location, URL: r.HTMLLink,
}
if e.Summary == "" {
// Google's own interface says this for an event with no title, and a blank
// chip in a grid reads as a rendering fault rather than as an empty title.
e.Summary = "(uden titel)"
}
switch {
case r.Start.Date != "":
e.AllDay = true
e.StartDay = r.Start.Date
// Google's all-day end is the day *after* the last one, so a one-day event
// ends tomorrow. Subtracting a day is what makes "covers this cell" a
// comparison rather than a special case at every call site.
e.EndDay = r.Start.Date
if day, err := time.Parse(dateLayout, r.End.Date); err == nil {
if last := day.AddDate(0, 0, -1); !last.Before(mustDay(r.Start.Date)) {
e.EndDay = last.Format(dateLayout)
}
}
case r.Start.DateTime != "":
e.StartsAt = r.Start.DateTime
e.EndsAt = r.End.DateTime
// The day is the date part of the timestamp Google returned, which carries
// the event's own offset. Parsing it into a time.Time and formatting it
// back would ask *this process* what day it is — and a container running in
// UTC would then move a 23:30 meeting in Copenhagen to the day before.
e.StartDay = dayOf(r.Start.DateTime)
e.EndDay = dayOf(r.End.DateTime)
if e.EndDay == "" {
e.EndDay = e.StartDay
}
// An event ending at midnight ends the evening before, as far as a grid is
// concerned. Without this, every meeting that runs to 24:00 puts a chip in
// the following day as well.
if e.EndDay > e.StartDay && strings.Contains(r.End.DateTime, "T00:00:00") {
if day, err := time.Parse(dateLayout, e.EndDay); err == nil {
e.EndDay = day.AddDate(0, 0, -1).Format(dateLayout)
}
}
default:
return Event{}, false
}
if e.EndDay < e.StartDay {
e.EndDay = e.StartDay
}
return e, true
}
// dayOf takes the date out of an RFC 3339 timestamp without interpreting it.
func dayOf(stamp string) string {
if len(stamp) < len(dateLayout) {
return ""
}
day := stamp[:len(dateLayout)]
if _, err := time.Parse(dateLayout, day); err != nil {
return ""
}
return day
}
func mustDay(s string) time.Time {
day, _ := time.Parse(dateLayout, s)
return day
}
func (c *Client) get(ctx context.Context, url string, out any) error {
return google.Get(ctx, c.http, c.accessToken, url, out)
}
+255
View File
@@ -0,0 +1,255 @@
package gcal
import (
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
// fakeGoogle answers the two calls this package makes, out of whatever the test
// hands it. There is no other way to exercise this: the real Calendar needs an
// account, a registered client and somebody's consent, and none of the three can be
// had inside a test — so the shapes Google returns are written down here and the
// parsing is checked against them.
func fakeGoogle(t *testing.T, calendarList any, events map[string]any) *Client {
t.Helper()
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
switch {
case strings.HasPrefix(r.URL.Path, "/users/me/calendarList"):
_ = json.NewEncoder(w).Encode(calendarList)
case strings.HasPrefix(r.URL.Path, "/calendars/"):
id := strings.TrimSuffix(strings.TrimPrefix(r.URL.Path, "/calendars/"), "/events")
body, ok := events[id]
if !ok {
http.Error(w, `{"error":"not found"}`, http.StatusNotFound)
return
}
_ = json.NewEncoder(w).Encode(body)
default:
http.Error(w, "unexpected path "+r.URL.Path, http.StatusNotFound)
}
}))
t.Cleanup(srv.Close)
return NewClient("token").At(srv.URL)
}
func TestCalendarsPrefersTheNameThePersonGaveIt(t *testing.T) {
client := fakeGoogle(t, map[string]any{
"items": []map[string]any{
{"id": "primary", "summary": "[email protected]", "backgroundColor": "#3f51b5",
"timeZone": "Europe/Copenhagen", "primary": true, "accessRole": "owner"},
{"id": "fam", "summary": "Family", "summaryOverride": "Familien",
"backgroundColor": "#0b8043", "accessRole": "reader"},
// A calendar removed from the list still arrives, flagged.
{"id": "gone", "summary": "Væk", "deleted": true, "accessRole": "reader"},
},
}, nil)
got, err := client.Calendars(t.Context())
if err != nil {
t.Fatal(err)
}
if len(got) != 2 {
t.Fatalf("got %d calendars, want 2 — a deleted one is not one you can show", len(got))
}
if got[1].Name != "Familien" {
t.Errorf("name = %q; summaryOverride is the name the person will look for", got[1].Name)
}
if !got[0].Primary || !got[0].Owner {
t.Errorf("the primary calendar came back as %+v", got[0])
}
if got[1].Owner {
t.Error("a reader was reported as an owner; nothing may offer to write to it")
}
if got[0].Colour != "#3f51b5" {
t.Errorf("colour = %q; two calendars in one grid are told apart by it", got[0].Colour)
}
}
// The three shapes an event arrives in, and the one shape a grid can read.
func TestEventsBecomeDaysAGridCanAskAbout(t *testing.T) {
client := fakeGoogle(t, nil, map[string]any{
"primary": map[string]any{
"items": []map[string]any{
{
"id": "timed", "summary": "Bestyrelsesmøde", "htmlLink": "https://cal/1",
"start": map[string]string{"dateTime": "2026-08-20T14:00:00+02:00"},
"end": map[string]string{"dateTime": "2026-08-20T15:30:00+02:00"},
},
{
// All-day, one day. Google's end is the day after.
"id": "allday", "summary": "Fridag",
"start": map[string]string{"date": "2026-08-21"},
"end": map[string]string{"date": "2026-08-22"},
},
{
// All-day, three days.
"id": "trip", "summary": "Aarhus",
"start": map[string]string{"date": "2026-08-24"},
"end": map[string]string{"date": "2026-08-27"},
},
{
// Runs to midnight, which is the evening before as far as a
// grid is concerned.
"id": "late", "summary": "Nattevagt",
"start": map[string]string{"dateTime": "2026-08-22T20:00:00+02:00"},
"end": map[string]string{"dateTime": "2026-08-23T00:00:00+02:00"},
},
{
// A cancelled occurrence of a recurring event.
"id": "off", "status": "cancelled", "summary": "Aflyst",
"start": map[string]string{"date": "2026-08-25"},
"end": map[string]string{"date": "2026-08-26"},
},
{
// Neither a date nor a dateTime: belongs on no day at all.
"id": "nowhere", "summary": "Uden tid",
},
},
},
})
got, err := client.Events(t.Context(), "primary", "2026-08-01", "2026-08-31")
if err != nil {
t.Fatal(err)
}
byID := map[string]Event{}
for _, e := range got {
byID[e.ID] = e
}
if len(got) != 4 {
t.Fatalf("got %d events, want 4: %+v", len(got), got)
}
if _, ok := byID["off"]; ok {
t.Error("a cancelled occurrence was kept; it is an event that is not happening")
}
if _, ok := byID["nowhere"]; ok {
t.Error("an event with no time was kept; there is no cell to draw it in")
}
timed := byID["timed"]
if timed.AllDay || timed.StartDay != "2026-08-20" || timed.EndDay != "2026-08-20" {
t.Errorf("timed = %+v", timed)
}
if timed.StartsAt != "2026-08-20T14:00:00+02:00" {
t.Errorf("the instant was rewritten: %q", timed.StartsAt)
}
allday := byID["allday"]
if !allday.AllDay || allday.StartDay != "2026-08-21" || allday.EndDay != "2026-08-21" {
t.Errorf("allday = %+v; Google's end date is exclusive and has to be corrected here",
allday)
}
if allday.StartsAt != "" {
t.Errorf("an all-day event was given an instant: %q — it has a day, not a moment",
allday.StartsAt)
}
if trip := byID["trip"]; trip.StartDay != "2026-08-24" || trip.EndDay != "2026-08-26" {
t.Errorf("trip = %+v, want 24th to 26th inclusive", trip)
}
if late := byID["late"]; late.EndDay != "2026-08-22" {
t.Errorf("late ends on %q; an event running to midnight ends the evening before, "+
"or every one of them puts a chip in the following day", late.EndDay)
}
}
// An event with no title has to say so. A blank chip in a grid reads as a
// rendering fault rather than as an empty title.
func TestAnEventWithNoTitleStillSaysSomething(t *testing.T) {
client := fakeGoogle(t, nil, map[string]any{
"c": map[string]any{"items": []map[string]any{{
"id": "x",
"start": map[string]string{"date": "2026-08-20"},
"end": map[string]string{"date": "2026-08-21"},
}}},
})
got, err := client.Events(t.Context(), "c", "2026-08-01", "2026-08-31")
if err != nil {
t.Fatal(err)
}
if len(got) != 1 || got[0].Summary == "" {
t.Fatalf("got %+v", got)
}
}
// The window is asked for with a day of slack at each end, because an event late
// in the evening in Copenhagen is the next day in UTC. The slack must not leak
// out: an event on the day before the window is not in the window.
func TestTheSlackAtTheEdgesIsThrownAwayAgain(t *testing.T) {
client := fakeGoogle(t, nil, map[string]any{
"c": map[string]any{"items": []map[string]any{
{"id": "before", "summary": "For tidligt",
"start": map[string]string{"date": "2026-08-09"},
"end": map[string]string{"date": "2026-08-10"}},
{"id": "inside", "summary": "Med",
"start": map[string]string{"date": "2026-08-10"},
"end": map[string]string{"date": "2026-08-11"}},
{"id": "after", "summary": "For sent",
"start": map[string]string{"date": "2026-08-21"},
"end": map[string]string{"date": "2026-08-22"}},
}},
})
got, err := client.Events(t.Context(), "c", "2026-08-10", "2026-08-20")
if err != nil {
t.Fatal(err)
}
if len(got) != 1 || got[0].ID != "inside" {
t.Fatalf("got %+v, want only the one inside the window", got)
}
}
// A page token has to be followed, or a busy calendar silently shows its first
// page and nothing else — which is a calendar that lies about a day being clear.
func TestEveryPageIsRead(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
body := map[string]any{
"items": []map[string]any{{"id": "a", "summary": "Første",
"start": map[string]string{"date": "2026-08-11"},
"end": map[string]string{"date": "2026-08-12"}}},
"nextPageToken": "more",
}
if r.URL.Query().Get("pageToken") == "more" {
body = map[string]any{"items": []map[string]any{{"id": "b", "summary": "Anden",
"start": map[string]string{"date": "2026-08-12"},
"end": map[string]string{"date": "2026-08-13"}}}}
}
_ = json.NewEncoder(w).Encode(body)
}))
defer srv.Close()
got, err := NewClient("t").At(srv.URL).Events(t.Context(), "c", "2026-08-01", "2026-08-31")
if err != nil {
t.Fatal(err)
}
if len(got) != 2 {
t.Fatalf("got %d events, want both pages: %+v", len(got), got)
}
}
// A 401 is not a failure to retry; it is a connection that has ended. Telling the
// two apart is what stops a revoked account being polled every ten minutes for ever.
func TestARefusedTokenIsItsOwnError(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.Error(w, `{"error":"invalid_credentials"}`, http.StatusUnauthorized)
}))
defer srv.Close()
_, err := NewClient("spent").At(srv.URL).Calendars(t.Context())
if err != ErrUnauthorized {
t.Fatalf("err = %v, want ErrUnauthorized", err)
}
}
// Read-only, and only the calendar. The consent screen is what somebody agrees to,
// and a scope that quietly grew is a promise quietly broken.
func TestTheScopeIsReadOnly(t *testing.T) {
if !strings.HasSuffix(Scope, "calendar.readonly") {
t.Errorf("scope = %q; nothing here writes a calendar", Scope)
}
}
+41 -213
View File
@@ -1,5 +1,5 @@
// Package gmail implements the OAuth2 authorisation-code flow with PKCE against
// Google, and the small slice of the Gmail API verdande needs.
// Package gmail is the small slice of the Gmail API verdande needs, and the scope
// it asks for.
//
// The connection is one-way on purpose: a starred or labelled message becomes a
// task, and nothing verdande does afterwards touches the mailbox. Unstarring the
@@ -9,200 +9,51 @@
//
// Scope is gmail.readonly. verdande never needs to send, modify or delete, and
// asking for less is what makes the consent screen something a person can agree to.
//
// The OAuth flow itself moved to internal/google when Calendar needed the same one.
// The names below are aliases so that every call site — and this package's own
// tests — still read as Gmail's, which is what they are: one registration, one
// flow, one scope each.
package gmail
import (
"context"
"crypto/rand"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"fmt"
"net/http"
"net/url"
"strings"
"time"
"github.com/kristianwind/verdande/internal/google"
)
const (
authEndpoint = "https://accounts.google.com/o/oauth2/v2/auth"
defaultTokenEndpoint = "https://oauth2.googleapis.com/token"
defaultAPIBase = "https://gmail.googleapis.com/gmail/v1"
// Read-only, and only mail. Not profile, not contacts, not send.
const Scope = "https://www.googleapis.com/auth/gmail.readonly"
// Read-only, and only mail. Not profile, not contacts, not send.
Scope = "https://www.googleapis.com/auth/gmail.readonly"
const defaultAPIBase = "https://gmail.googleapis.com/gmail/v1"
// The OAuth flow, which is Google's rather than Gmail's.
type (
Config = google.Config
PKCE = google.PKCE
Token = google.Token
)
// Endpoints is where Google is.
//
// A field rather than a constant so a test can point it at a server it controls.
// There is no other way to exercise what happens when Gmail is slow or refuses —
// and "the sync comes back before whatever sits in front of this server gives up
// on it" is exactly the kind of promise worth a test and impossible to make
// against the real thing.
//
// The zero value is Google, so nothing outside a test has to know it exists.
type Endpoints struct {
Token string
API string
}
func (e Endpoints) token() string {
if e.Token != "" {
return e.Token
}
return defaultTokenEndpoint
}
func (e Endpoints) api() string {
if e.API != "" {
return e.API
}
return defaultAPIBase
}
// Config is the OAuth client the operator registered in Google Cloud. It belongs to
// the instance rather than to a person: one registration, any number of users.
type Config struct {
ClientID string
ClientSecret string
RedirectURL string
// Endpoints is Google unless a test says otherwise.
Endpoints Endpoints
}
func (c Config) Configured() bool {
return c.ClientID != "" && c.ClientSecret != "" && c.RedirectURL != ""
}
// PKCE is the proof-key pair for one authorisation attempt.
//
// PKCE on a server-side client is not strictly required — the client secret already
// authenticates the token exchange — but it costs nothing and closes the window in
// which an authorisation code intercepted from the redirect could be replayed
// before verdande gets to it.
type PKCE struct {
Verifier string
Challenge string
State string
}
func NewPKCE() (PKCE, error) {
verifier, err := randomString(64)
if err != nil {
return PKCE{}, err
}
state, err := randomString(32)
if err != nil {
return PKCE{}, err
}
sum := sha256.Sum256([]byte(verifier))
return PKCE{
Verifier: verifier,
Challenge: base64.RawURLEncoding.EncodeToString(sum[:]),
State: state,
}, nil
}
// AuthURL is where the browser is sent to ask the person for consent.
func (c Config) AuthURL(p PKCE) string {
v := url.Values{}
v.Set("client_id", c.ClientID)
v.Set("redirect_uri", c.RedirectURL)
v.Set("response_type", "code")
v.Set("scope", Scope)
v.Set("state", p.State)
v.Set("code_challenge", p.Challenge)
v.Set("code_challenge_method", "S256")
// Without both of these Google returns no refresh token on a re-authorisation,
// and the connection silently stops working an hour later.
v.Set("access_type", "offline")
v.Set("prompt", "consent")
return authEndpoint + "?" + v.Encode()
}
type Token struct {
AccessToken string
RefreshToken string
ExpiresAt time.Time
}
// Exchange trades the authorisation code for tokens.
func (c Config) Exchange(ctx context.Context, code string, p PKCE) (Token, error) {
form := url.Values{}
form.Set("code", code)
form.Set("client_id", c.ClientID)
form.Set("client_secret", c.ClientSecret)
form.Set("redirect_uri", c.RedirectURL)
form.Set("grant_type", "authorization_code")
form.Set("code_verifier", p.Verifier)
return c.token(ctx, form)
}
// Refresh gets a fresh access token. Google does not return a new refresh token
// here, so the caller keeps the one it has.
func (c Config) Refresh(ctx context.Context, refreshToken string) (Token, error) {
form := url.Values{}
form.Set("refresh_token", refreshToken)
form.Set("client_id", c.ClientID)
form.Set("client_secret", c.ClientSecret)
form.Set("grant_type", "refresh_token")
token, err := c.token(ctx, form)
if err != nil {
return Token{}, err
}
token.RefreshToken = refreshToken
return token, nil
}
func (c Config) token(ctx context.Context, form url.Values) (Token, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.Endpoints.token(),
strings.NewReader(form.Encode()))
if err != nil {
return Token{}, err
}
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
resp, err := (&http.Client{Timeout: 30 * time.Second}).Do(req)
if err != nil {
return Token{}, err
}
defer resp.Body.Close()
var parsed struct {
AccessToken string `json:"access_token"`
RefreshToken string `json:"refresh_token"`
ExpiresIn int `json:"expires_in"`
Error string `json:"error"`
ErrorDescription string `json:"error_description"`
}
if err := json.NewDecoder(resp.Body).Decode(&parsed); err != nil {
return Token{}, fmt.Errorf("gmail: reading token response: %w", err)
}
if parsed.Error != "" {
return Token{}, fmt.Errorf("gmail: %s: %s", parsed.Error, parsed.ErrorDescription)
}
if resp.StatusCode != http.StatusOK || parsed.AccessToken == "" {
return Token{}, fmt.Errorf("gmail: token exchange failed with %s", resp.Status)
}
return Token{
AccessToken: parsed.AccessToken,
RefreshToken: parsed.RefreshToken,
// A minute of slack, so a token that is about to expire is refreshed
// rather than used and rejected.
ExpiresAt: time.Now().Add(time.Duration(parsed.ExpiresIn-60) * time.Second),
}, nil
}
var (
NewPKCE = google.NewPKCE
ErrUnauthorized = google.ErrUnauthorized
)
// --- the API ---------------------------------------------------------------------
type Client struct {
accessToken string
endpoints Endpoints
http *http.Client
// base is where the Gmail API is. A field rather than a constant so a test can
// point it at a server it controls — and "the sync comes back before whatever
// sits in front of this server gives up on it" is exactly the kind of promise
// worth a test and impossible to make against the real thing. Empty is Google.
base string
http *http.Client
}
func NewClient(accessToken string) *Client {
@@ -210,11 +61,18 @@ func NewClient(accessToken string) *Client {
}
// At points the client at another server. Tests only; the zero value is Google.
func (c *Client) At(e Endpoints) *Client {
c.endpoints = e
func (c *Client) At(base string) *Client {
c.base = base
return c
}
func (c *Client) api() string {
if c.base != "" {
return c.base
}
return defaultAPIBase
}
// Message is the part of a Gmail message a task is made from.
type Message struct {
ID string
@@ -233,7 +91,7 @@ func (c *Client) Profile(ctx context.Context) (string, error) {
var parsed struct {
EmailAddress string `json:"emailAddress"`
}
if err := c.get(ctx, c.endpoints.api()+"/users/me/profile", &parsed); err != nil {
if err := c.get(ctx, c.api()+"/users/me/profile", &parsed); err != nil {
return "", err
}
return parsed.EmailAddress, nil
@@ -257,7 +115,7 @@ func (c *Client) List(ctx context.Context, query string, max int) ([]string, err
ID string `json:"id"`
} `json:"messages"`
}
if err := c.get(ctx, c.endpoints.api()+"/users/me/messages?"+v.Encode(), &parsed); err != nil {
if err := c.get(ctx, c.api()+"/users/me/messages?"+v.Encode(), &parsed); err != nil {
return nil, err
}
@@ -288,7 +146,7 @@ func (c *Client) Get(ctx context.Context, id string) (Message, error) {
} `json:"headers"`
} `json:"payload"`
}
if err := c.get(ctx, c.endpoints.api()+"/users/me/messages/"+url.PathEscape(id)+"?"+v.Encode(), &parsed); err != nil {
if err := c.get(ctx, c.api()+"/users/me/messages/"+url.PathEscape(id)+"?"+v.Encode(), &parsed); err != nil {
return Message{}, err
}
@@ -308,31 +166,9 @@ func (c *Client) Get(ctx context.Context, id string) (Message, error) {
}
func (c *Client) get(ctx context.Context, url string, out any) error {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+c.accessToken)
resp, err := c.http.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode == http.StatusUnauthorized {
return ErrUnauthorized
}
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("gmail: %s", resp.Status)
}
return json.NewDecoder(resp.Body).Decode(out)
return google.Get(ctx, c.http, c.accessToken, url, out)
}
// ErrUnauthorized means the tokens are no longer good — the person revoked access,
// or changed their password. The caller disconnects rather than retrying.
var ErrUnauthorized = fmt.Errorf("gmail: authorisation was refused")
// SenderName pulls "Anders Jensen" out of "Anders Jensen <[email protected]>",
// falling back to the address when there is no display name.
func SenderName(from string) string {
@@ -375,11 +211,3 @@ func Query(trigger, label string) string {
terms = append(terms, "newer_than:30d")
return strings.Join(terms, " ")
}
func randomString(n int) (string, error) {
b := make([]byte, n)
if _, err := rand.Read(b); err != nil {
return "", err
}
return base64.RawURLEncoding.EncodeToString(b)[:n], nil
}
+4 -98
View File
@@ -1,113 +1,19 @@
package gmail
import (
"crypto/sha256"
"encoding/base64"
"net/url"
"strings"
"testing"
)
// The challenge has to be the SHA-256 of the verifier, base64url without padding.
// Getting this wrong means Google refuses every exchange with a message that does
// not say which end is at fault.
func TestPKCEChallengeMatchesItsVerifier(t *testing.T) {
p, err := NewPKCE()
if err != nil {
t.Fatal(err)
}
sum := sha256.Sum256([]byte(p.Verifier))
want := base64.RawURLEncoding.EncodeToString(sum[:])
if p.Challenge != want {
t.Errorf("challenge = %q, want %q", p.Challenge, want)
}
// RFC 7636 requires 43–128 characters, and no padding.
if n := len(p.Verifier); n < 43 || n > 128 {
t.Errorf("verifier is %d characters, outside the 43–128 the spec allows", n)
}
if strings.ContainsAny(p.Verifier, "=+/") {
t.Errorf("verifier %q contains characters that are not URL-safe", p.Verifier)
}
if p.State == "" {
t.Error("no state was generated")
}
// Two attempts must not share anything.
other, err := NewPKCE()
if err != nil {
t.Fatal(err)
}
if other.Verifier == p.Verifier || other.State == p.State {
t.Error("two authorisation attempts produced the same secrets")
}
}
func TestAuthURL(t *testing.T) {
cfg := Config{
ClientID: "client-123",
ClientSecret: "secret",
RedirectURL: "https://todo.example.dk/oauth/gmail/callback",
}
p, err := NewPKCE()
if err != nil {
t.Fatal(err)
}
raw := cfg.AuthURL(p)
parsed, err := url.Parse(raw)
if err != nil {
t.Fatalf("the URL does not parse: %v", err)
}
q := parsed.Query()
checks := map[string]string{
"client_id": cfg.ClientID,
"redirect_uri": cfg.RedirectURL,
"response_type": "code",
"scope": Scope,
"code_challenge": p.Challenge,
"code_challenge_method": "S256",
"state": p.State,
// Without both of these Google returns no refresh token on a
// re-authorisation, and the connection dies silently an hour later.
"access_type": "offline",
"prompt": "consent",
}
for key, want := range checks {
if got := q.Get(key); got != want {
t.Errorf("%s = %q, want %q", key, got, want)
}
}
// The verifier itself must never leave the server.
if strings.Contains(raw, p.Verifier) {
t.Error("the code verifier was put in the authorisation URL")
}
// Read-only, and only mail.
// The scope is the one thing verdande chooses about the connection, and it is the
// one the person reads on the consent screen. The flow itself is exercised in
// internal/google; what belongs here is that Gmail still asks for read-only mail.
func TestTheScopeIsReadOnlyMail(t *testing.T) {
if !strings.HasSuffix(Scope, "gmail.readonly") {
t.Errorf("scope = %q; verdande never needs to send or modify", Scope)
}
}
func TestConfigured(t *testing.T) {
full := Config{ClientID: "a", ClientSecret: "b", RedirectURL: "c"}
if !full.Configured() {
t.Error("a complete config is not reported as configured")
}
for _, partial := range []Config{
{},
{ClientID: "a"},
{ClientID: "a", ClientSecret: "b"},
{ClientSecret: "b", RedirectURL: "c"},
} {
if partial.Configured() {
t.Errorf("%+v was reported as configured", partial)
}
}
}
func TestQuery(t *testing.T) {
cases := []struct {
trigger, label string
+224
View File
@@ -0,0 +1,224 @@
// Package google implements the OAuth2 authorisation-code flow with PKCE against
// Google.
//
// It was written inside internal/gmail, because Gmail was the only thing that
// needed it. Calendar needs the same flow — the same endpoints, the same PKCE, the
// same refresh — and the only thing that differs between them is one string: the
// scope. A second copy would be a second place for "access_type=offline" to be
// forgotten, which is the mistake that makes a connection die silently an hour
// after it was made.
//
// What is *not* here is the API calls. Those differ completely, so gmail and gcal
// keep their own clients and share only the way in.
package google
import (
"context"
"crypto/rand"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"fmt"
"net/http"
"net/url"
"strings"
"time"
)
const (
authEndpoint = "https://accounts.google.com/o/oauth2/v2/auth"
defaultTokenEndpoint = "https://oauth2.googleapis.com/token"
)
// Config is the OAuth client the operator registered in Google Cloud, plus the
// scope one feature is asking for.
//
// The client belongs to the instance rather than to a person: one registration,
// any number of users. The scope belongs to the feature: Gmail and Calendar sign
// in through the same registration and ask for different things, and Google issues
// a refresh token per authorisation — so connecting one does not disturb the other.
type Config struct {
ClientID string
ClientSecret string
RedirectURL string
Scope string
// TokenURL is where Google is. A field rather than a constant so a test can
// point it at a server it controls: there is no other way to exercise what
// happens when the token endpoint is slow or refuses. Empty is Google, so
// nothing outside a test has to know it exists.
TokenURL string
}
func (c Config) Configured() bool {
return c.ClientID != "" && c.ClientSecret != "" && c.RedirectURL != ""
}
func (c Config) tokenURL() string {
if c.TokenURL != "" {
return c.TokenURL
}
return defaultTokenEndpoint
}
// PKCE is the proof-key pair for one authorisation attempt.
//
// PKCE on a server-side client is not strictly required — the client secret already
// authenticates the token exchange — but it costs nothing and closes the window in
// which an authorisation code intercepted from the redirect could be replayed
// before verdande gets to it.
type PKCE struct {
Verifier string
Challenge string
State string
}
func NewPKCE() (PKCE, error) {
verifier, err := randomString(64)
if err != nil {
return PKCE{}, err
}
state, err := randomString(32)
if err != nil {
return PKCE{}, err
}
sum := sha256.Sum256([]byte(verifier))
return PKCE{
Verifier: verifier,
Challenge: base64.RawURLEncoding.EncodeToString(sum[:]),
State: state,
}, nil
}
// AuthURL is where the browser is sent to ask the person for consent.
func (c Config) AuthURL(p PKCE) string {
v := url.Values{}
v.Set("client_id", c.ClientID)
v.Set("redirect_uri", c.RedirectURL)
v.Set("response_type", "code")
v.Set("scope", c.Scope)
v.Set("state", p.State)
v.Set("code_challenge", p.Challenge)
v.Set("code_challenge_method", "S256")
// Without both of these Google returns no refresh token on a re-authorisation,
// and the connection silently stops working an hour later.
v.Set("access_type", "offline")
v.Set("prompt", "consent")
return authEndpoint + "?" + v.Encode()
}
type Token struct {
AccessToken string
RefreshToken string
ExpiresAt time.Time
}
// Exchange trades the authorisation code for tokens.
func (c Config) Exchange(ctx context.Context, code string, p PKCE) (Token, error) {
form := url.Values{}
form.Set("code", code)
form.Set("client_id", c.ClientID)
form.Set("client_secret", c.ClientSecret)
form.Set("redirect_uri", c.RedirectURL)
form.Set("grant_type", "authorization_code")
form.Set("code_verifier", p.Verifier)
return c.token(ctx, form)
}
// Refresh gets a fresh access token. Google does not return a new refresh token
// here, so the caller keeps the one it has.
func (c Config) Refresh(ctx context.Context, refreshToken string) (Token, error) {
form := url.Values{}
form.Set("refresh_token", refreshToken)
form.Set("client_id", c.ClientID)
form.Set("client_secret", c.ClientSecret)
form.Set("grant_type", "refresh_token")
token, err := c.token(ctx, form)
if err != nil {
return Token{}, err
}
token.RefreshToken = refreshToken
return token, nil
}
func (c Config) token(ctx context.Context, form url.Values) (Token, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.tokenURL(),
strings.NewReader(form.Encode()))
if err != nil {
return Token{}, err
}
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
resp, err := (&http.Client{Timeout: 30 * time.Second}).Do(req)
if err != nil {
return Token{}, err
}
defer resp.Body.Close()
var parsed struct {
AccessToken string `json:"access_token"`
RefreshToken string `json:"refresh_token"`
ExpiresIn int `json:"expires_in"`
Error string `json:"error"`
ErrorDescription string `json:"error_description"`
}
if err := json.NewDecoder(resp.Body).Decode(&parsed); err != nil {
return Token{}, fmt.Errorf("google: reading token response: %w", err)
}
if parsed.Error != "" {
return Token{}, fmt.Errorf("google: %s: %s", parsed.Error, parsed.ErrorDescription)
}
if resp.StatusCode != http.StatusOK || parsed.AccessToken == "" {
return Token{}, fmt.Errorf("google: token exchange failed with %s", resp.Status)
}
return Token{
AccessToken: parsed.AccessToken,
RefreshToken: parsed.RefreshToken,
// A minute of slack, so a token that is about to expire is refreshed
// rather than used and rejected.
ExpiresAt: time.Now().Add(time.Duration(parsed.ExpiresIn-60) * time.Second),
}, nil
}
// ErrUnauthorized means the tokens are no longer good — the person revoked access,
// or changed their password. The caller disconnects rather than retrying.
var ErrUnauthorized = fmt.Errorf("google: authorisation was refused")
// Get performs an authorised GET and decodes the JSON body into out.
//
// Shared by the two API clients because the interesting part is the same in both:
// a 401 is not a failure to retry, it is a connection that has ended, and telling
// those apart is what stops a revoked account being polled every ten minutes for
// ever.
func Get(ctx context.Context, client *http.Client, accessToken, url string, out any) error {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+accessToken)
resp, err := client.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode == http.StatusUnauthorized {
return ErrUnauthorized
}
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("google: %s", resp.Status)
}
return json.NewDecoder(resp.Body).Decode(out)
}
func randomString(n int) (string, error) {
b := make([]byte, n)
if _, err := rand.Read(b); err != nil {
return "", err
}
return base64.RawURLEncoding.EncodeToString(b)[:n], nil
}
+127
View File
@@ -0,0 +1,127 @@
package google
import (
"crypto/sha256"
"encoding/base64"
"net/url"
"strings"
"testing"
)
// The challenge has to be the SHA-256 of the verifier, base64url without padding.
// Getting this wrong means Google refuses every exchange with a message that does
// not say which end is at fault.
func TestPKCEChallengeMatchesItsVerifier(t *testing.T) {
p, err := NewPKCE()
if err != nil {
t.Fatal(err)
}
sum := sha256.Sum256([]byte(p.Verifier))
want := base64.RawURLEncoding.EncodeToString(sum[:])
if p.Challenge != want {
t.Errorf("challenge = %q, want %q", p.Challenge, want)
}
// RFC 7636 requires 43–128 characters, and no padding.
if n := len(p.Verifier); n < 43 || n > 128 {
t.Errorf("verifier is %d characters, outside the 43–128 the spec allows", n)
}
if strings.ContainsAny(p.Verifier, "=+/") {
t.Errorf("verifier %q contains characters that are not URL-safe", p.Verifier)
}
if p.State == "" {
t.Error("no state was generated")
}
// Two attempts must not share anything.
other, err := NewPKCE()
if err != nil {
t.Fatal(err)
}
if other.Verifier == p.Verifier || other.State == p.State {
t.Error("two authorisation attempts produced the same secrets")
}
}
func TestAuthURL(t *testing.T) {
cfg := Config{
ClientID: "client-123",
ClientSecret: "secret",
RedirectURL: "https://todo.example.dk/oauth/gmail/callback",
Scope: "https://www.googleapis.com/auth/gmail.readonly",
}
p, err := NewPKCE()
if err != nil {
t.Fatal(err)
}
raw := cfg.AuthURL(p)
parsed, err := url.Parse(raw)
if err != nil {
t.Fatalf("the URL does not parse: %v", err)
}
q := parsed.Query()
checks := map[string]string{
"client_id": cfg.ClientID,
"redirect_uri": cfg.RedirectURL,
"response_type": "code",
"scope": cfg.Scope,
"code_challenge": p.Challenge,
"code_challenge_method": "S256",
"state": p.State,
// Without both of these Google returns no refresh token on a
// re-authorisation, and the connection dies silently an hour later.
"access_type": "offline",
"prompt": "consent",
}
for key, want := range checks {
if got := q.Get(key); got != want {
t.Errorf("%s = %q, want %q", key, got, want)
}
}
// The verifier itself must never leave the server.
if strings.Contains(raw, p.Verifier) {
t.Error("the code verifier was put in the authorisation URL")
}
}
// The scope is the one thing that differs between the two features signing in
// through this, so it has to come out of the config rather than out of a constant
// somebody could forget to change.
func TestTheScopeComesFromTheConfig(t *testing.T) {
p, err := NewPKCE()
if err != nil {
t.Fatal(err)
}
calendar := "https://www.googleapis.com/auth/calendar.readonly"
raw := Config{ClientID: "a", Scope: calendar}.AuthURL(p)
parsed, err := url.Parse(raw)
if err != nil {
t.Fatal(err)
}
if got := parsed.Query().Get("scope"); got != calendar {
t.Errorf("scope = %q, want %q — asking for the wrong one is a consent screen "+
"that grants the wrong thing", got, calendar)
}
}
func TestConfigured(t *testing.T) {
full := Config{ClientID: "a", ClientSecret: "b", RedirectURL: "c"}
if !full.Configured() {
t.Error("a complete config is not reported as configured")
}
for _, partial := range []Config{
{},
{ClientID: "a"},
{ClientID: "a", ClientSecret: "b"},
{ClientSecret: "b", RedirectURL: "c"},
} {
if partial.Configured() {
t.Errorf("%+v was reported as configured", partial)
}
}
}
+72 -2
View File
@@ -32,6 +32,7 @@ type userJSON struct {
// about the work and not about the screen — the sidebar's width is the
// opposite case and stays in localStorage.
SidebarCollapsed []string `json:"sidebar_collapsed"`
NavOrder []string `json:"nav_order"`
CreatedAt string `json:"created_at"`
}
@@ -42,10 +43,14 @@ func toUserJSON(u *store.User) userJSON {
if collapsed == nil {
collapsed = []string{}
}
order := u.NavOrder
if order == nil {
order = []string{}
}
return userJSON{
ID: u.ID, Email: u.Email, Name: u.Name, AvatarColor: u.AvatarColor,
Timezone: u.Timezone, Locale: u.Locale, IsAdmin: u.IsAdmin,
TOTPEnabled: u.TOTPEnabled, SidebarCollapsed: collapsed,
TOTPEnabled: u.TOTPEnabled, SidebarCollapsed: collapsed, NavOrder: order,
CreatedAt: u.CreatedAt.Format(time.RFC3339),
}
}
@@ -83,6 +88,30 @@ func (s *Server) handleSetSidebarSections(w http.ResponseWriter, r *http.Request
w.WriteHeader(http.StatusNoContent)
}
// handleSetNavOrder records the order of the fixed views in the sidebar.
//
// Its own route for the same reason as the folds: this writes on every drop, and
// sharing the profile's path would make a reorder send a name and a timezone it
// was never asked to change.
func (s *Server) handleSetNavOrder(w http.ResponseWriter, r *http.Request) {
var req struct {
Order []string `json:"order"`
}
if err := decodeJSON(w, r, &req); err != nil {
return
}
if len(req.Order) > 20 {
writeError(w, http.StatusBadRequest, CodeBadRequest, "too many entries")
return
}
user := userFrom(r.Context())
if err := s.db.SetNavOrder(r.Context(), user.ID, req.Order); err != nil {
s.storeError(w, r, "set nav order", err)
return
}
w.WriteHeader(http.StatusNoContent)
}
// --- sign in -----------------------------------------------------------------
type loginRequest struct {
@@ -125,7 +154,21 @@ func (s *Server) handleLogin(w http.ResponseWriter, r *http.Request) {
}
}
s.loginLimiter.reset(clientIP(r))
// The bucket is cleared only when the login is *finished*.
//
// It used to be cleared here, on the strength of the password alone — before
// the second factor had been asked for at all. That handed somebody who
// already had the password an unlimited supply of ten-guess batches: log in,
// get a fresh bucket and a fresh pending session, spend ten guesses on the
// code, log in again. At three valid codes in any moment and twenty requests a
// second, that is even odds inside an afternoon — against the one control that
// exists precisely for when the password is already gone.
//
// So an account with a second factor clears nothing yet. handleLoginTOTP does
// it, once the code is right.
if !user.TOTPEnabled {
s.loginLimiter.reset(clientIP(r))
}
// A session that still needs a code gets a short life of its own: long enough
// to fetch a phone, not long enough to be worth stealing.
@@ -162,6 +205,19 @@ func (s *Server) handleLoginTOTP(w http.ResponseWriter, r *http.Request) {
user := userFrom(r.Context())
session := sessionFrom(r.Context())
// Counted per account as well as per address.
//
// The address bucket alone is defeated by anybody with more than one, which is
// the ordinary case — a phone and a laptop is already two. A code is six
// digits: the only thing that makes it safe is that guesses are scarce, and
// scarcity has to be counted against the thing being attacked.
if ok, retry := s.totpLimiter.allow(user.ID); !ok {
w.Header().Set("Retry-After", retryAfterSeconds(retry))
writeError(w, http.StatusTooManyRequests, CodeRateLimited,
"too many verification codes; wait a moment and try again")
return
}
err := auth.VerifyTOTP(user.TOTPSecret, req.Code, time.Now())
if err != nil {
// A recovery code is accepted in the same field. Making somebody find a
@@ -169,6 +225,15 @@ func (s *Server) handleLoginTOTP(w http.ResponseWriter, r *http.Request) {
// introduce a second concept.
used, rerr := s.db.UseRecoveryCode(r.Context(), user.ID, req.Code)
if rerr != nil || !used {
// The pending session is left alive on purpose.
//
// Ending it on the first wrong digit was considered and rejected: it
// would make somebody who mistyped one number type their password again,
// and it buys almost nothing. What makes a six-digit code safe is that
// guesses are scarce, and the per-account bucket above is what makes
// them scarce — ten an hour, however many times the password is
// re-entered. The attack this closes was never "one session, many
// guesses"; it was "a fresh bucket for every password check".
writeError(w, http.StatusUnauthorized, CodeUnauthorized,
"that verification code is not valid")
return
@@ -176,6 +241,11 @@ func (s *Server) handleLoginTOTP(w http.ResponseWriter, r *http.Request) {
s.log.Info("recovery code used", "user", user.ID)
}
// Finished. Now the buckets may be cleared — both of them, so somebody who
// fumbled the code twice is not still waiting once they get it right.
s.loginLimiter.reset(clientIP(r))
s.totpLimiter.reset(user.ID)
if err := s.db.PromoteSession(r.Context(), session.ID, s.cfg.SessionTTL); err != nil {
s.internal(w, r, "promote session", err)
return
+70 -2
View File
@@ -301,7 +301,11 @@ func TestTOTPEnrolmentAndTwoStepLogin(t *testing.T) {
// Begin: a secret exists, but 2FA is not on yet — closing the tab here must
// not lock the account.
resp, body := ts.do(t, "POST", "/api/v1/auth/totp/setup", nil)
// Med adgangskoden: at slå andenfaktoren til er mindst lige så indgribende som
// at slå den fra, og den har altid spurgt.
resp, body := ts.do(t, "POST", "/api/v1/auth/totp/setup", map[string]string{
"password": "et langt kodeord",
})
if resp.StatusCode != http.StatusOK {
t.Fatalf("totp setup: %d %v", resp.StatusCode, body)
}
@@ -368,7 +372,9 @@ func TestRecoveryCodeCompletesLoginOnce(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
_, body := ts.do(t, "POST", "/api/v1/auth/totp/setup", nil)
_, body := ts.do(t, "POST", "/api/v1/auth/totp/setup", map[string]string{
"password": "et langt kodeord",
})
secret := body["secret"].(string)
_, body = ts.do(t, "POST", "/api/v1/auth/totp/confirm", map[string]string{
"code": codeFor(t, secret),
@@ -897,3 +903,65 @@ func TestUserAgentsAreSummarisedForAPerson(t *testing.T) {
}
}
}
// Andenfaktoren kan ikke slås til uden adgangskoden.
//
// Døren åbnede før med et token og lukkede kun med et kodeord: `disable` har
// altid spurgt, `setup` spurgte om ingenting. Et stjålet token kunne derfor hente
// hemmeligheden, lægge den i sin egen godkender, bekræfte den og beholde den
// eneste kopi af gendannelseskoderne — hvorefter ejerens næste login stoppede ved
// en kode, de ikke havde, og de ikke kunne slå den fra uden at logge ind først.
func TestTOTPCannotBeSwitchedOnWithoutThePassword(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
resp, _ := ts.do(t, "POST", "/api/v1/auth/totp/setup", map[string]string{
"password": "det forkerte kodeord",
})
if resp.StatusCode != http.StatusUnprocessableEntity {
t.Fatalf("status %d, ventede at en forkert adgangskode blev afvist", resp.StatusCode)
}
resp, _ = ts.do(t, "POST", "/api/v1/auth/totp/setup", map[string]string{})
if resp.StatusCode == http.StatusOK {
t.Fatal("en tom adgangskode blev accepteret")
}
}
// Ratebegrænseren må ikke nulstilles af adgangskoden alene.
//
// Den blev det, og det gav én, der havde kodeordet, ubegrænsede portioner på ti
// gæt: log ind, få en frisk spand og en frisk ventende session, brug ti gæt på
// koden, log ind igen. Andenfaktoren findes præcis til det tilfælde, hvor
// kodeordet allerede er tabt, så den portion skal betales for hver gang.
func TestAPasswordAloneDoesNotRefillTheCodeAttempts(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
_, body := ts.do(t, "POST", "/api/v1/auth/totp/setup", map[string]string{
"password": "et langt kodeord",
})
secret := body["secret"].(string)
ts.do(t, "POST", "/api/v1/auth/totp/confirm", map[string]string{"code": codeFor(t, secret)})
ts.do(t, "POST", "/api/v1/auth/logout", nil)
// Kodeordet er rigtigt hver gang; kun koden er forkert. Uden loftet ville det
// her kunne blive ved i det uendelige.
refused := 0
for i := 0; i < 14; i++ {
resp, _ := ts.do(t, "POST", "/api/v1/auth/login", map[string]string{
"email": "[email protected]", "password": "et langt kodeord",
})
if resp.StatusCode == http.StatusTooManyRequests {
refused++
continue
}
resp, _ = ts.do(t, "POST", "/api/v1/auth/login/totp", map[string]string{"code": "000000"})
if resp.StatusCode == http.StatusTooManyRequests {
refused++
}
}
if refused == 0 {
t.Fatal("fjorten runder med forkerte koder blev aldrig afvist; spanden fyldes stadig af adgangskoden")
}
}
+297
View File
@@ -0,0 +1,297 @@
package httpapi
import (
"context"
"net/http"
"strings"
"time"
"github.com/kristianwind/verdande/internal/beacon"
"github.com/kristianwind/verdande/internal/safedial"
"github.com/kristianwind/verdande/internal/store"
)
// The beacon, from the instance's side.
//
// Two halves that happen to live in the same binary: the part that reports in,
// which every installation runs, and the part that receives, which only the one
// acting as collector turns on. See internal/beacon for what is sent and why it
// is on by default rather than compulsory.
const beaconScope = "beacon"
// beaconConfig is what this instance has been told to do. The zero value is what
// a fresh installation gets, which is why Enabled defaults to true elsewhere and
// not here — a zero struct must never mean "silently on".
type beaconConfig struct {
Enabled bool `json:"enabled"`
Collector string `json:"collector_url"`
InstanceID string `json:"instance_id"`
IsCollector bool `json:"is_collector"`
PublishCount bool `json:"publish_count"`
LastPingAt int64 `json:"last_ping_at,omitempty"`
}
// beaconSettings reads the configuration, filling in the defaults and minting the
// anonymous id on first use.
//
// The id is generated here rather than derived from anything about the machine.
// A hash of the hostname or the base URL would be stable and convenient and would
// also be a value the collector could compare against a guess — which is the
// difference between anonymous and pseudonymous, and it is the whole point.
func (s *Server) beaconSettings(ctx context.Context) (beaconConfig, error) {
cfg := beaconConfig{Enabled: true, Collector: beacon.DefaultCollector}
stored, err := s.db.InstanceSettings(ctx, beaconScope)
if err != nil {
return cfg, err
}
if v, ok := stored["enabled"].(bool); ok {
cfg.Enabled = v
}
if v, ok := stored["collector_url"].(string); ok && strings.TrimSpace(v) != "" {
cfg.Collector = strings.TrimSpace(v)
}
if v, ok := stored["instance_id"].(string); ok && beacon.ValidID(v) {
cfg.InstanceID = v
}
if v, ok := stored["is_collector"].(bool); ok {
cfg.IsCollector = v
}
if v, ok := stored["publish_count"].(bool); ok {
cfg.PublishCount = v
}
if v, ok := stored["last_ping_at"].(float64); ok {
cfg.LastPingAt = int64(v)
}
if cfg.InstanceID == "" {
cfg.InstanceID = store.NewID()
if err := s.saveBeaconSettings(ctx, cfg); err != nil {
return cfg, err
}
}
return cfg, nil
}
func (s *Server) saveBeaconSettings(ctx context.Context, cfg beaconConfig) error {
return s.db.SetInstanceSettings(ctx, beaconScope, map[string]any{
"enabled": cfg.Enabled,
"collector_url": cfg.Collector,
"instance_id": cfg.InstanceID,
"is_collector": cfg.IsCollector,
"publish_count": cfg.PublishCount,
"last_ping_at": cfg.LastPingAt,
})
}
// --- what the settings page shows ------------------------------------------------
type beaconStatusJSON struct {
Enabled bool `json:"enabled"`
Collector string `json:"collector_url"`
InstanceID string `json:"instance_id"`
Version string `json:"version"`
IsCollector bool `json:"is_collector"`
PublishCount bool `json:"publish_count"`
LastPingAt string `json:"last_ping_at,omitempty"`
Stats *store.BeaconStats `json:"stats,omitempty"`
}
func (s *Server) handleBeaconStatus(w http.ResponseWriter, r *http.Request) {
cfg, err := s.beaconSettings(r.Context())
if err != nil {
s.internal(w, r, "beacon settings", err)
return
}
out := beaconStatusJSON{
Enabled: cfg.Enabled, Collector: cfg.Collector, InstanceID: cfg.InstanceID,
Version: Version, IsCollector: cfg.IsCollector, PublishCount: cfg.PublishCount,
}
if cfg.LastPingAt > 0 {
out.LastPingAt = time.Unix(cfg.LastPingAt, 0).Format(time.RFC3339)
}
if cfg.IsCollector {
stats, err := s.db.BeaconSummary(r.Context())
if err != nil {
s.internal(w, r, "beacon summary", err)
return
}
out.Stats = &stats
}
writeJSON(w, http.StatusOK, out)
}
type beaconRequest struct {
Enabled *bool `json:"enabled"`
Collector *string `json:"collector_url"`
IsCollector *bool `json:"is_collector"`
PublishCount *bool `json:"publish_count"`
}
func (s *Server) handleSetBeacon(w http.ResponseWriter, r *http.Request) {
var req beaconRequest
if err := decodeJSON(w, r, &req); err != nil {
return
}
cfg, err := s.beaconSettings(r.Context())
if err != nil {
s.internal(w, r, "beacon settings", err)
return
}
if req.Enabled != nil {
cfg.Enabled = *req.Enabled
}
if req.IsCollector != nil {
cfg.IsCollector = *req.IsCollector
}
if req.PublishCount != nil {
cfg.PublishCount = *req.PublishCount
}
if req.Collector != nil {
url := strings.TrimSpace(*req.Collector)
if url == "" {
url = beacon.DefaultCollector
}
// http and https only, and said as a field error rather than a 500: an
// operator pointing this at their own collector will mistype it, and the
// page should say which field is wrong.
if !strings.HasPrefix(url, "http://") && !strings.HasPrefix(url, "https://") {
writeFieldErrors(w, map[string]string{"collector_url": "must start with http:// or https://"})
return
}
cfg.Collector = url
}
if err := s.saveBeaconSettings(r.Context(), cfg); err != nil {
s.internal(w, r, "save beacon settings", err)
return
}
s.handleBeaconStatus(w, r)
}
// --- the collector's side ---------------------------------------------------------
// handleBeaconPing receives one report. Unauthenticated, because the whole point
// is that a stranger's installation can reach it.
//
// Nothing about the request is recorded except the two values in the body. The
// source address is not read here, and there is no code path from this function
// to anything that could write one down — which is the only version of that
// promise worth making.
func (s *Server) handleBeaconPing(w http.ResponseWriter, r *http.Request) {
cfg, err := s.beaconSettings(r.Context())
if err != nil {
s.internal(w, r, "beacon settings", err)
return
}
// An instance that is not a collector does not quietly accept and drop pings —
// it says there is nothing here. Otherwise every installation in the world is a
// spam sink that answers 200 to anything.
if !cfg.IsCollector {
writeError(w, http.StatusNotFound, CodeNotFound, "this instance is not a beacon collector")
return
}
r.Body = http.MaxBytesReader(w, r.Body, 4<<10)
var p beacon.Payload
if err := decodeJSON(w, r, &p); err != nil {
return
}
// Checked, not trusted. This is the one endpoint anybody can post to, and the
// two columns behind it should only ever hold the two shapes this program
// writes — an id that is not id-shaped is somebody probing, not an install.
if !beacon.ValidID(p.InstanceID) {
writeFieldErrors(w, map[string]string{"instance_id": "malformed"})
return
}
if p.Version != "" && !beacon.ValidVersion(p.Version) {
writeFieldErrors(w, map[string]string{"version": "malformed"})
return
}
if err := s.db.RecordBeacon(r.Context(), p.InstanceID, p.Version); err != nil {
s.internal(w, r, "record beacon", err)
return
}
w.WriteHeader(http.StatusNoContent)
}
// handleBeaconCount publishes the number, when the collector has been told to.
//
// Thirty days rather than a running total: a total only climbs, so it stops
// meaning "how many people use this" and starts meaning "how many ever tried it".
func (s *Server) handleBeaconCount(w http.ResponseWriter, r *http.Request) {
cfg, err := s.beaconSettings(r.Context())
if err != nil {
s.internal(w, r, "beacon settings", err)
return
}
if !cfg.IsCollector || !cfg.PublishCount {
writeError(w, http.StatusNotFound, CodeNotFound, "not published")
return
}
n, err := s.db.BeaconCount(r.Context(), 30*24*time.Hour)
if err != nil {
s.internal(w, r, "beacon count", err)
return
}
w.Header().Set("Cache-Control", "public, max-age=60")
writeJSON(w, http.StatusOK, map[string]any{"installs": n, "window_days": 30})
}
// SendBeacon reports this installation, at most once a day.
//
// Exported and handed to the jobs runner the same way the mailbox syncs are: the
// settings, the anonymous id and the version all live on this side, and the
// runner should not have to know about any of them.
//
// The day is counted from the last successful send, not from a wall-clock hour.
// A fleet of installations that all pinged at 03:00 would arrive in one spike;
// counted this way they spread themselves out by when each was started.
func (s *Server) SendBeacon(ctx context.Context) error {
cfg, err := s.beaconSettings(ctx)
if err != nil {
return err
}
if !cfg.Enabled {
return nil
}
// The collector does not report to itself over the network. It would work, and
// it would also mean the one instance whose count matters most depends on its
// own tunnel being up to be counted.
if cfg.IsCollector {
if err := s.db.RecordBeacon(ctx, cfg.InstanceID, Version); err != nil {
return err
}
cfg.LastPingAt = time.Now().Unix()
return s.saveBeaconSettings(ctx, cfg)
}
if time.Since(time.Unix(cfg.LastPingAt, 0)) < 24*time.Hour {
return nil
}
// safedial, ikke http.DefaultClient. Adressen kan skrives i fladen, og en
// administrator, der peger den på 127.0.0.1 eller på 169.254.169.254, ville
// ellers have en maskine, der henter indefra på kommando — og svaret på om den
// svarede, er i sig selv det, sådan et forsøg leder efter.
if err := beacon.Send(ctx, safedial.Client(15*time.Second), cfg.Collector, beacon.Payload{
InstanceID: cfg.InstanceID,
Version: Version,
}); err != nil {
// Logged at info, not error. The collector being unreachable is not this
// installation's problem and must never look like one — it is somebody
// else's server, and the only consequence is a number being one lower.
s.log.Info("beacon not delivered", "err", err)
return nil
}
cfg.LastPingAt = time.Now().Unix()
return s.saveBeaconSettings(ctx, cfg)
}
+548
View File
@@ -0,0 +1,548 @@
package httpapi
import (
"context"
"errors"
"net/http"
"time"
"github.com/kristianwind/verdande/internal/gcal"
"github.com/kristianwind/verdande/internal/store"
)
// defaultCalendarSyncBudget bounds one "fetch now" when nothing else says
// otherwise. Well under the hundred seconds a proxy is willing to wait, so this
// server is the one that answers a slow Google rather than Cloudflare's own HTML
// page — which is how a failure arrives with no code, no message and nothing in
// the error log.
const defaultCalendarSyncBudget = 25 * time.Second
// The window verdande keeps a copy of.
//
// A month grid pages, so asking Google on every arrow press would mean a round trip
// for a question answered ninety seconds ago — a calendar slower than the calendar
// it is showing. A cached window is the alternative, and a window has to end
// somewhere. Ninety days back is roughly "when was that", a year forward is past
// anything anybody has booked; between them they cover every screen the grid can
// reach without a person noticing an edge.
//
// The edge is still real, and the events endpoint says where it is rather than
// answering an empty list for a month outside it. A calendar that quietly shows no
// events is a calendar that lies about a day being clear.
const (
calendarWindowBack = 90
calendarWindowAhead = 365
calendarDateLayout = "2006-01-02"
calendarProviderName = "google"
)
func (s *Server) calendarClient(accessToken string) *gcal.Client {
return gcal.NewClient(accessToken).At(s.cfg.CalendarAPIURL)
}
// calendarConfig is the same registration Gmail signs in through, asking for a
// different scope and coming back to a different URI.
//
// One OAuth client, two features. Google issues a refresh token per authorisation
// and keeps up to a hundred of them live per account, each carrying the scopes it
// was granted with — so connecting the calendar does not disturb a working Gmail,
// and disconnecting one does not take the other with it.
func (s *Server) calendarConfig(ctx context.Context) gcal.Config {
cfg := s.gmailConfig(ctx)
cfg.Scope = gcal.Scope
cfg.RedirectURL = s.cfg.CalendarRedirectURL()
return cfg
}
// --- connecting ------------------------------------------------------------------
// handleCalendarAuthorize starts the flow. Same shape as Gmail's, and the same
// reason for keeping the verifier server-side: the callback arrives as a top-level
// navigation from Google, and a SameSite=Lax cookie is the kind of thing that works
// until a browser tightens its defaults.
func (s *Server) handleCalendarAuthorize(w http.ResponseWriter, r *http.Request) {
user := userFrom(r.Context())
cfg := s.calendarConfig(r.Context())
if !cfg.Configured() {
writeError(w, http.StatusConflict, CodeGmailNotConfigured,
"no Google OAuth client is registered on this server")
return
}
pkce, err := gcal.NewPKCE()
if err != nil {
s.internal(w, r, "calendar pkce", err)
return
}
// Its own settings scope, not Gmail's. Two handshakes in one row would mean a
// person who started both flows had one verifier, and whichever came back
// second would be refused for a reason nothing on the screen could explain.
settings, err := s.db.UserSettings(r.Context(), user.ID, "calendar")
if err != nil {
s.internal(w, r, "calendar settings", err)
return
}
settings["pkce_verifier"] = pkce.Verifier
settings["pkce_state"] = pkce.State
// The attempt expires: a stored verifier that lives forever is a replay window
// that lives forever.
settings["pkce_expires"] = time.Now().Add(10 * time.Minute).Unix()
if err := s.db.SetUserSettings(r.Context(), user.ID, "calendar", settings); err != nil {
s.internal(w, r, "calendar settings", err)
return
}
writeJSON(w, http.StatusOK, map[string]string{"url": cfg.AuthURL(pkce)})
}
// handleCalendarCallback is where Google sends the browser back.
//
// It answers with a redirect into the app rather than with JSON: the person is
// looking at a browser tab, not at a fetch response.
func (s *Server) handleCalendarCallback(w http.ResponseWriter, r *http.Request) {
user := userFrom(r.Context())
settingsURL := s.cfg.BaseURL + "/indstillinger/integrationer"
if errParam := r.URL.Query().Get("error"); errParam != "" {
// Somebody clicked "cancel" on the consent screen — or, on an Internal
// OAuth app, Google refused an account outside the organisation with
// `org_internal`. Both arrive here, and the page names what it was given.
http.Redirect(w, r, settingsURL+"?calendar="+errParam, http.StatusFound)
return
}
code := r.URL.Query().Get("code")
state := r.URL.Query().Get("state")
if code == "" || state == "" {
http.Redirect(w, r, settingsURL+"?calendar=invalid", http.StatusFound)
return
}
settings, err := s.db.UserSettings(r.Context(), user.ID, "calendar")
if err != nil {
s.internal(w, r, "calendar settings", err)
return
}
verifier, _ := settings["pkce_verifier"].(string)
expected, _ := settings["pkce_state"].(string)
expires, _ := settings["pkce_expires"].(float64)
// The state has to match, or this endpoint accepts a code from anywhere.
if verifier == "" || expected == "" || state != expected {
http.Redirect(w, r, settingsURL+"?calendar=state", http.StatusFound)
return
}
if int64(expires) < time.Now().Unix() {
http.Redirect(w, r, settingsURL+"?calendar=expired", http.StatusFound)
return
}
token, err := s.calendarConfig(r.Context()).Exchange(r.Context(), code,
gcal.PKCE{Verifier: verifier})
if err != nil {
s.log.Error("calendar exchange", "err", err, "user", user.ID)
http.Redirect(w, r, settingsURL+"?calendar=failed", http.StatusFound)
return
}
if token.RefreshToken == "" {
// Without one the connection dies in an hour. Google only withholds it when
// the consent prompt was skipped, which access_type and prompt are meant to
// prevent — so this means something is wrong with the client registration
// rather than with this request.
s.log.Error("calendar returned no refresh token", "user", user.ID)
http.Redirect(w, r, settingsURL+"?calendar=norefresh", http.StatusFound)
return
}
// The one-time values are cleared: keeping a spent verifier serves nothing and
// is one more secret at rest.
delete(settings, "pkce_verifier")
delete(settings, "pkce_state")
delete(settings, "pkce_expires")
if err := s.db.SetUserSettings(r.Context(), user.ID, "calendar", settings); err != nil {
s.internal(w, r, "clear calendar handshake", err)
return
}
account := &store.CalendarAccount{
UserID: user.ID, Provider: calendarProviderName,
RefreshToken: token.RefreshToken, AccessToken: token.AccessToken,
ExpiresAt: token.ExpiresAt,
}
if err := s.db.SaveCalendarAccount(r.Context(), account); err != nil {
s.internal(w, r, "save calendar tokens", err)
return
}
// The list of calendars, fetched now so the page that opens next has something
// to tick rather than an empty panel and a button. Best effort: an account that
// connects but cannot list itself is still connected, and the status endpoint
// asks again when it finds nothing.
if _, err := s.refreshCalendarList(r.Context(), account); err != nil {
s.log.Warn("calendar list", "err", err, "user", user.ID)
}
s.log.Info("calendar connected", "user", user.ID, "account", account.Account)
http.Redirect(w, r, settingsURL+"?calendar=connected", http.StatusFound)
}
// --- what the settings page reads --------------------------------------------------
type calendarStatusJSON struct {
Connected bool `json:"connected"`
Account string `json:"account,omitempty"`
Calendars []store.Calendar `json:"calendars"`
// RedirectURI is what has to be registered with Google, spelled out. Derived
// from VERDANDE_BASE_URL and matched exactly — the single most likely thing to
// be wrong, and the error Google gives for it names neither value.
RedirectURI string `json:"redirect_uri"`
// HasClient says whether there is an OAuth registration to connect through at
// all, so the page can explain rather than offer a button that answers 409.
HasClient bool `json:"has_client"`
LastSyncAt string `json:"last_sync_at,omitempty"`
// ReadOnly is stated rather than assumed. Nothing writes a Google calendar, and
// an interface that quietly looks editable is one somebody will try to edit.
ReadOnly bool `json:"read_only"`
}
func (s *Server) handleGetCalendar(w http.ResponseWriter, r *http.Request) {
user := userFrom(r.Context())
out := calendarStatusJSON{
Calendars: []store.Calendar{},
RedirectURI: s.cfg.CalendarRedirectURL(),
HasClient: s.calendarConfig(r.Context()).Configured(),
ReadOnly: true,
}
account, err := s.db.CalendarAccountFor(r.Context(), user.ID, calendarProviderName)
if err != nil {
s.internal(w, r, "calendar account", err)
return
}
if account == nil || account.RefreshToken == "" {
writeJSON(w, http.StatusOK, out)
return
}
out.Connected = true
out.Account = account.Account
if !account.LastSyncAt.IsZero() {
out.LastSyncAt = account.LastSyncAt.Format(time.RFC3339)
}
calendars, err := s.db.Calendars(r.Context(), account.ID)
if err != nil {
s.internal(w, r, "calendars", err)
return
}
// An account connected during a Google hiccup has no calendars and no way back:
// the person sees an empty list and nothing to press. Asking again here costs
// one request on a page nobody opens often, and means the wrong answer lasts
// until the next look rather than for ever. Same reasoning as the Gmail address.
if len(calendars) == 0 {
if fetched, err := s.refreshCalendarList(r.Context(), account); err == nil {
calendars = fetched
out.Account = account.Account
} else {
s.log.Warn("calendar list", "err", err, "user", user.ID)
}
}
if calendars != nil {
out.Calendars = calendars
}
writeJSON(w, http.StatusOK, out)
}
type showCalendarsRequest struct {
// Shown is the whole set, not one flip. Same reasoning as the projects' order:
// a person has a handful of calendars, the question is "these ones", and a list
// that can only be built by several requests can land half applied.
Shown []string `json:"shown"`
}
func (s *Server) handleSetCalendars(w http.ResponseWriter, r *http.Request) {
var req showCalendarsRequest
if err := decodeJSON(w, r, &req); err != nil {
return
}
user := userFrom(r.Context())
account, err := s.db.CalendarAccountFor(r.Context(), user.ID, calendarProviderName)
if err != nil {
s.internal(w, r, "calendar account", err)
return
}
if account == nil {
// A 404 rather than a 409: everything the caller may not see answers 404,
// and "there is no such connection" is the true answer either way.
writeError(w, http.StatusNotFound, CodeNotFound, "no calendar is connected")
return
}
if err := s.db.ShowCalendars(r.Context(), account.ID, req.Shown); err != nil {
s.internal(w, r, "show calendars", err)
return
}
w.WriteHeader(http.StatusNoContent)
}
// handleDisconnectCalendar forgets the connection, and the copy of the calendars
// with it. Unlike the tasks a mailbox made, none of this is the person's own work.
func (s *Server) handleDisconnectCalendar(w http.ResponseWriter, r *http.Request) {
user := userFrom(r.Context())
account, err := s.db.CalendarAccountFor(r.Context(), user.ID, calendarProviderName)
if err != nil {
s.internal(w, r, "calendar account", err)
return
}
if account != nil {
if err := s.db.DeleteCalendarAccount(r.Context(), user.ID, account.ID); err != nil {
s.internal(w, r, "disconnect calendar", err)
return
}
}
w.WriteHeader(http.StatusNoContent)
}
// --- what the grid reads -----------------------------------------------------------
type calendarEventsJSON struct {
Events []store.CalendarEvent `json:"events"`
// From and To are the window verdande holds a copy of, not the window that was
// asked for. A grid that has paged past it gets no events, and an empty answer
// with no explanation is a calendar quietly claiming a day is clear.
From string `json:"from"`
To string `json:"to"`
}
func (s *Server) handleCalendarEvents(w http.ResponseWriter, r *http.Request) {
user := userFrom(r.Context())
from := r.URL.Query().Get("from")
to := r.URL.Query().Get("to")
fields := map[string]string{}
if !validDay(from) {
fields["from"] = "must be a date, as 2026-08-20"
}
if !validDay(to) {
fields["to"] = "must be a date, as 2026-08-20"
}
if len(fields) > 0 {
writeFieldErrors(w, fields)
return
}
events, err := s.db.CalendarEvents(r.Context(), user.ID, from, to)
if err != nil {
s.internal(w, r, "calendar events", err)
return
}
if events == nil {
events = []store.CalendarEvent{}
}
windowFrom, windowTo := calendarWindow(time.Now())
writeJSON(w, http.StatusOK, calendarEventsJSON{
Events: events, From: windowFrom, To: windowTo,
})
}
// handleCalendarSyncNow refreshes immediately, so somebody who has just connected
// can see it work rather than wondering whether it did.
func (s *Server) handleCalendarSyncNow(w http.ResponseWriter, r *http.Request) {
user := userFrom(r.Context())
// A budget of its own, well under what sits in front of this server. An unset
// budget is not a budget of nothing: a zero here would expire the context
// before the first call and make every sync a silent no-op.
budget := s.cfg.CalendarSyncBudget
if budget <= 0 {
budget = defaultCalendarSyncBudget
}
ctx, cancel := context.WithTimeout(r.Context(), budget)
defer cancel()
count, err := s.SyncCalendars(ctx, user)
if err != nil {
if errors.Is(err, context.DeadlineExceeded) {
// Not a failure: Google is slow or busy, some calendars were read, and
// the rest arrive on the next run. Every calendar is committed as it is
// read, so a short budget is a shorter run rather than a lost one.
writeJSON(w, http.StatusOK, map[string]any{"events": count, "partial": true})
return
}
s.upstream(w, r, CodeCalendarFailed, "sync calendar", err)
return
}
writeJSON(w, http.StatusOK, map[string]int{"events": count})
}
// SyncCalendars refreshes every calendar this person has chosen to see.
//
// Returns how many events were written. Also used by the background job, which is
// why it takes a context and a user rather than a request.
func (s *Server) SyncCalendars(ctx context.Context, user *store.User) (int, error) {
account, err := s.db.CalendarAccountFor(ctx, user.ID, calendarProviderName)
if err != nil {
return 0, err
}
if account == nil || account.RefreshToken == "" {
return 0, nil // not connected; nothing to do and nothing to complain about
}
// The same guard the mailboxes have, for the same reason: a press of "fetch
// now" during the sweep would otherwise have two runs replacing the same window
// at once, and the loser's delete would land after the winner's insert.
unlock := s.lockSync("calendar:" + account.ID)
defer unlock()
if fresh, err := s.db.CalendarAccountFor(ctx, user.ID, calendarProviderName); err == nil && fresh != nil {
account = fresh
}
access, err := s.calendarAccessToken(ctx, account)
if err != nil {
return 0, err
}
client := s.calendarClient(access)
// The list first, so a calendar added or renamed since the last run is drawn
// under its new name and one that has gone stops being drawn at all.
calendars, err := s.calendarList(ctx, client, account)
if err != nil {
return 0, s.forgetIfRevoked(ctx, user, account, err)
}
from, to := calendarWindow(time.Now())
written := 0
for _, calendar := range calendars {
if !calendar.Shown {
continue
}
// A spent budget ends the run rather than turning into a failed fetch per
// calendar, logged one at a time.
if ctx.Err() != nil {
return written, ctx.Err()
}
events, err := client.Events(ctx, calendar.RemoteID, from, to)
if err != nil {
if errors.Is(err, gcal.ErrUnauthorized) {
return written, s.forgetIfRevoked(ctx, user, account, err)
}
// One calendar somebody was unshared from must not stop the others.
s.log.Warn("calendar events", "err", err, "calendar", calendar.Name)
continue
}
rows := make([]store.CalendarEvent, 0, len(events))
for _, e := range events {
rows = append(rows, store.CalendarEvent{
RemoteID: e.ID, Summary: e.Summary,
StartsAt: e.StartsAt, EndsAt: e.EndsAt,
StartDay: e.StartDay, EndDay: e.EndDay,
AllDay: e.AllDay, Location: e.Location, URL: e.URL,
})
}
if err := s.db.ReplaceCalendarEvents(ctx, calendar.ID, from, to, rows); err != nil {
return written, err
}
written += len(rows)
}
if err := s.db.MarkCalendarSynced(ctx, account.ID, time.Now()); err != nil {
return written, err
}
return written, nil
}
// calendarAccessToken returns a usable access token, refreshing only when the one
// held has actually expired.
func (s *Server) calendarAccessToken(ctx context.Context, account *store.CalendarAccount) (string, error) {
if account.AccessToken != "" && account.ExpiresAt.After(time.Now()) {
return account.AccessToken, nil
}
token, err := s.calendarConfig(ctx).Refresh(ctx, account.RefreshToken)
if err != nil {
return "", err
}
account.AccessToken = token.AccessToken
account.ExpiresAt = token.ExpiresAt
if err := s.db.SaveCalendarAccount(ctx, account); err != nil {
return "", err
}
return account.AccessToken, nil
}
// refreshCalendarList fetches the calendar list with a token it obtains itself.
// Used by the paths that have an account but no client yet — connecting, and the
// status endpoint correcting an empty list.
func (s *Server) refreshCalendarList(ctx context.Context, account *store.CalendarAccount) ([]store.Calendar, error) {
access, err := s.calendarAccessToken(ctx, account)
if err != nil {
return nil, err
}
return s.calendarList(ctx, s.calendarClient(access), account)
}
// calendarList writes what the provider says and returns what is stored, which is
// not the same list: the provider says what exists, and only verdande knows which
// of them somebody chose to look at.
func (s *Server) calendarList(ctx context.Context, client *gcal.Client, account *store.CalendarAccount) ([]store.Calendar, error) {
remote, err := client.Calendars(ctx)
if err != nil {
return nil, err
}
list := make([]store.Calendar, 0, len(remote))
for _, c := range remote {
list = append(list, store.Calendar{
RemoteID: c.ID, Name: c.Name, Colour: c.Colour,
TimeZone: c.TimeZone, Primary: c.Primary, Writable: c.Owner,
})
// The primary calendar is the account, and its id is the address. It is the
// only place the connected account names itself without a second scope —
// asking for a profile scope to learn an address already in the answer would
// be one more thing on the consent screen for nothing.
if c.Primary && account.Account == "" {
account.Account = c.ID
if err := s.db.SaveCalendarAccount(ctx, account); err != nil {
return nil, err
}
}
}
if err := s.db.ReplaceCalendars(ctx, account.ID, list); err != nil {
return nil, err
}
return s.db.Calendars(ctx, account.ID)
}
// forgetIfRevoked disconnects when Google says the grant is gone, and passes every
// other failure through untouched.
//
// The distinction is the point. A revoked account polled every ten minutes for ever
// is a log line a week long and a connection the person believes they removed; a
// Google that is merely down is one that will answer next time.
func (s *Server) forgetIfRevoked(ctx context.Context, user *store.User, account *store.CalendarAccount, err error) error {
if errors.Is(err, gcal.ErrUnauthorized) {
s.log.Info("calendar access was revoked; disconnecting", "user", user.ID)
_ = s.db.DeleteCalendarAccount(ctx, user.ID, account.ID)
}
return err
}
// calendarWindow is the span verdande keeps a copy of, measured from today.
//
// Built from the calendar date rather than by truncating the instant: Truncate
// rounds against the Unix epoch, which is midnight UTC, and in Copenhagen that is
// two in the morning — so for two hours of every summer day the window would start
// and end a day out.
func calendarWindow(now time.Time) (string, string) {
day := time.Date(now.Year(), now.Month(), now.Day(), 0, 0, 0, 0, now.Location())
return day.AddDate(0, 0, -calendarWindowBack).Format(calendarDateLayout),
day.AddDate(0, 0, calendarWindowAhead).Format(calendarDateLayout)
}
func validDay(s string) bool {
_, err := time.Parse(calendarDateLayout, s)
return err == nil
}
+471
View File
@@ -0,0 +1,471 @@
package httpapi
import (
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"github.com/kristianwind/verdande/internal/config"
"github.com/kristianwind/verdande/internal/store"
)
// fakeCalendarGoogle stands in for the parts of Google this feature touches: the
// token endpoint, the calendar list and one calendar's events.
//
// There is no other way to exercise any of it. The real thing needs a registered
// OAuth client, an account and somebody's consent, and none of the three can be had
// inside a test — so the shapes Google returns are written down and the behaviour
// is checked against them, the way internal/imap is checked against go-imap's own
// in-process server.
type fakeCalendarGoogle struct {
*httptest.Server
// refreshes counts token refreshes, so a test can prove one is not made for a
// token that has not expired.
refreshes int
// refuse makes every API call answer 401, which is what a revoked grant does.
refuse bool
events map[string][]map[string]any
}
func newFakeCalendarGoogle(t *testing.T) *fakeCalendarGoogle {
t.Helper()
g := &fakeCalendarGoogle{events: map[string][]map[string]any{}}
g.Server = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
switch {
case r.URL.Path == "/token":
g.refreshes++
_ = json.NewEncoder(w).Encode(map[string]any{
"access_token": "ya29.fresh", "refresh_token": "1//0-refresh",
"expires_in": 3600,
})
case g.refuse:
http.Error(w, `{"error":"invalid_credentials"}`, http.StatusUnauthorized)
case strings.HasPrefix(r.URL.Path, "/users/me/calendarList"):
_ = json.NewEncoder(w).Encode(map[string]any{"items": []map[string]any{
{"id": "[email protected]", "summary": "[email protected]", "primary": true,
"accessRole": "owner", "backgroundColor": "#3f51b5",
"timeZone": "Europe/Copenhagen"},
{"id": "fam", "summary": "Familien", "accessRole": "reader",
"backgroundColor": "#0b8043"},
}})
case strings.HasPrefix(r.URL.Path, "/calendars/"):
id := strings.TrimSuffix(strings.TrimPrefix(r.URL.Path, "/calendars/"), "/events")
items := g.events[id]
if items == nil {
items = []map[string]any{}
}
_ = json.NewEncoder(w).Encode(map[string]any{"items": items})
default:
http.Error(w, "unexpected "+r.URL.Path, http.StatusNotFound)
}
}))
t.Cleanup(g.Close)
return g
}
// connectedCalendar returns a server with a calendar account already connected —
// the state the OAuth callback leaves behind, without the consent screen nothing
// can automate.
func connectedCalendar(t *testing.T, g *fakeCalendarGoogle, tune func(*config.Config)) (*testServer, *store.User) {
t.Helper()
ts := newTestServerWith(t, func(cfg *config.Config) {
cfg.GmailClientID = "test-client"
cfg.GmailClientSecret = "test-secret"
cfg.GoogleTokenURL = g.URL + "/token"
cfg.CalendarAPIURL = g.URL
if tune != nil {
tune(cfg)
}
})
ts.bootstrap(t)
user, err := ts.db.UserByEmail(t.Context(), "[email protected]")
if err != nil {
t.Fatal(err)
}
account := &store.CalendarAccount{
UserID: user.ID, Provider: "google", RefreshToken: "1//0-refresh",
}
if err := ts.db.SaveCalendarAccount(t.Context(), account); err != nil {
t.Fatal(err)
}
return ts, user
}
// The scope on the consent screen is what somebody agrees to. It has to be the
// read-only one, and the redirect has to be the calendar's own — sharing Gmail's
// would put "which token store do I write" behind a lookup.
func TestAuthorizeAsksForReadOnlyCalendar(t *testing.T) {
g := newFakeCalendarGoogle(t)
ts, _ := connectedCalendar(t, g, nil)
resp, body := ts.do(t, "POST", "/api/v1/calendar/authorize", nil)
if resp.StatusCode != http.StatusOK {
t.Fatalf("status %d, body %v", resp.StatusCode, body)
}
url, _ := body["url"].(string)
for _, want := range []string{
"scope=" + "https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcalendar.readonly",
"%2Foauth%2Fcalendar%2Fcallback",
"access_type=offline",
"prompt=consent",
} {
if !strings.Contains(url, want) {
t.Errorf("the authorisation URL is missing %q:\n%s", want, url)
}
}
if strings.Contains(url, "calendar.events&") || strings.Contains(url, "auth%2Fcalendar&") {
t.Errorf("a writing scope was asked for:\n%s", url)
}
}
// Without a registered client there is nothing to connect through, and the answer
// has to say that rather than "that already exists".
func TestAuthorizeWithoutAClientSaysSo(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
resp, body := ts.do(t, "POST", "/api/v1/calendar/authorize", nil)
if resp.StatusCode != http.StatusConflict {
t.Fatalf("status %d, want 409", resp.StatusCode)
}
if body["code"] != CodeGmailNotConfigured {
t.Errorf("code = %v, want %q", body["code"], CodeGmailNotConfigured)
}
}
// The whole path, minus the consent screen: sync fetches the list, refreshes the
// events of what was chosen, and the grid endpoint answers with them.
func TestSyncFetchesTheChosenCalendarsAndTheGridReadsThem(t *testing.T) {
g := newFakeCalendarGoogle(t)
g.events["fam"] = []map[string]any{
{"id": "e1", "summary": "Bestyrelsesmøde", "htmlLink": "https://calendar.google.com/e1",
"start": map[string]string{"dateTime": "2026-08-20T14:00:00+02:00"},
"end": map[string]string{"dateTime": "2026-08-20T15:30:00+02:00"}},
}
g.events["[email protected]"] = []map[string]any{
{"id": "e2", "summary": "Skal ikke med",
"start": map[string]string{"date": "2026-08-21"},
"end": map[string]string{"date": "2026-08-22"}},
}
ts, user := connectedCalendar(t, g, nil)
// The list arrives on the first look at the settings page, which is what
// somebody would be doing at this point.
resp, body := ts.do(t, "GET", "/api/v1/calendar", nil)
if resp.StatusCode != http.StatusOK {
t.Fatalf("status %d, body %v", resp.StatusCode, body)
}
if body["connected"] != true {
t.Fatalf("not reported as connected: %v", body)
}
if body["read_only"] != true {
t.Error("the connection did not say it is read-only; an interface that looks " +
"editable is one somebody will try to edit")
}
// The primary calendar's id is the address, which is how the account names
// itself without asking for a profile scope as well.
if body["account"] != "[email protected]" {
t.Errorf("account = %v", body["account"])
}
list, _ := body["calendars"].([]any)
if len(list) != 2 {
t.Fatalf("got %d calendars, want 2: %v", len(list), body["calendars"])
}
var famID string
for _, raw := range list {
c := raw.(map[string]any)
if c["shown"] == true {
t.Errorf("%v was shown without anybody choosing it", c["name"])
}
if c["remote_id"] == "fam" {
famID = c["id"].(string)
}
}
// Nothing is chosen, so a sync writes nothing: an account with a dozen shared
// calendars must not fill somebody's grid the moment it is connected.
resp, body = ts.do(t, "POST", "/api/v1/calendar/sync", nil)
if resp.StatusCode != http.StatusOK || body["events"] != float64(0) {
t.Fatalf("an unchosen calendar was fetched: %d %v", resp.StatusCode, body)
}
resp, _ = ts.do(t, "PUT", "/api/v1/calendar/calendars",
map[string]any{"shown": []string{famID}})
if resp.StatusCode != http.StatusNoContent {
t.Fatalf("choosing a calendar: status %d", resp.StatusCode)
}
resp, body = ts.do(t, "POST", "/api/v1/calendar/sync", nil)
if resp.StatusCode != http.StatusOK {
t.Fatalf("sync: status %d, body %v", resp.StatusCode, body)
}
if body["events"] != float64(1) {
t.Fatalf("events = %v, want 1 — only the chosen calendar", body["events"])
}
resp, body = ts.do(t, "GET",
"/api/v1/calendar/events?from=2026-08-01&to=2026-08-31", nil)
if resp.StatusCode != http.StatusOK {
t.Fatalf("events: status %d, body %v", resp.StatusCode, body)
}
events, _ := body["events"].([]any)
if len(events) != 1 {
t.Fatalf("got %d events, want 1: %v", len(events), body["events"])
}
event := events[0].(map[string]any)
if event["summary"] != "Bestyrelsesmøde" || event["start_day"] != "2026-08-20" {
t.Errorf("the event came back as %v", event)
}
// The calendar's colour rides along, or the chip has to ask for it per row.
if event["colour"] != "#0b8043" {
t.Errorf("colour = %v; two calendars in one grid are told apart by it", event["colour"])
}
// The window verdande holds is stated, so a grid paged past it can say so
// rather than showing an empty month it cannot explain.
if body["from"] == "" || body["to"] == "" {
t.Error("the answer does not say which window it covers")
}
// And it is one person's calendar. Somebody else on the same instance sees
// nothing of it.
other := ts.newUser(t, "[email protected]", "Andreas")
_, body = other.do(t, "GET", "/api/v1/calendar/events?from=2026-08-01&to=2026-08-31", nil)
if events, _ := body["events"].([]any); len(events) != 0 {
t.Errorf("somebody else's calendar was visible: %v", body["events"])
}
_ = user
}
// A token that has not expired must not be traded in for another. Refreshing on
// every run burns the quota and makes the failure mode of a bad refresh token
// arrive on every sweep rather than once an hour.
func TestAFreshTokenIsNotRefreshed(t *testing.T) {
g := newFakeCalendarGoogle(t)
ts, user := connectedCalendar(t, g, nil)
account, err := ts.db.CalendarAccountFor(t.Context(), user.ID, "google")
if err != nil {
t.Fatal(err)
}
account.AccessToken = "ya29.still-good"
account.ExpiresAt = time.Now().Add(time.Hour)
if err := ts.db.SaveCalendarAccount(t.Context(), account); err != nil {
t.Fatal(err)
}
if resp, body := ts.do(t, "POST", "/api/v1/calendar/sync", nil); resp.StatusCode != http.StatusOK {
t.Fatalf("sync: status %d, body %v", resp.StatusCode, body)
}
if g.refreshes != 0 {
t.Errorf("the token was refreshed %d times while it was still good", g.refreshes)
}
// And an expired one is.
account.ExpiresAt = time.Now().Add(-time.Minute)
if err := ts.db.SaveCalendarAccount(t.Context(), account); err != nil {
t.Fatal(err)
}
if resp, body := ts.do(t, "POST", "/api/v1/calendar/sync", nil); resp.StatusCode != http.StatusOK {
t.Fatalf("sync: status %d, body %v", resp.StatusCode, body)
}
if g.refreshes != 1 {
t.Errorf("an expired token was refreshed %d times, want 1", g.refreshes)
}
}
// Access revoked at Google's end is a connection that has ended, not a failure to
// retry. Polling it every fifteen minutes for ever is a week of log lines and a
// connection the person believes they removed.
func TestRevokedAccessDisconnects(t *testing.T) {
g := newFakeCalendarGoogle(t)
ts, user := connectedCalendar(t, g, nil)
account, _ := ts.db.CalendarAccountFor(t.Context(), user.ID, "google")
account.AccessToken = "ya29.revoked"
account.ExpiresAt = time.Now().Add(time.Hour)
if err := ts.db.SaveCalendarAccount(t.Context(), account); err != nil {
t.Fatal(err)
}
g.refuse = true
resp, body := ts.do(t, "POST", "/api/v1/calendar/sync", nil)
if resp.StatusCode != StatusUpstreamRefused {
t.Fatalf("status %d, want %d — Google saying no is not this server breaking",
resp.StatusCode, StatusUpstreamRefused)
}
if body["code"] != CodeCalendarFailed {
t.Errorf("code = %v, want %q — `internal_error` is shown as a generic sentence, "+
"which throws the diagnosis away", body["code"], CodeCalendarFailed)
}
gone, err := ts.db.CalendarAccountFor(t.Context(), user.ID, "google")
if err != nil {
t.Fatal(err)
}
if gone != nil {
t.Error("the connection is still there after Google refused the token")
}
// And the failure is in the error log, which is the point of having one: a
// sync that has been failing for a week is invisible otherwise.
_, logged := ts.do(t, "GET", "/api/v1/errors", nil)
rows, _ := logged["errors"].([]any)
if len(rows) == 0 {
t.Fatal("nothing was recorded")
}
if row := rows[0].(map[string]any); row["what"] != "sync calendar" {
t.Errorf("what = %v, want \"sync calendar\"", row["what"])
}
}
// Disconnecting takes the copy with it. None of it is the person's own work, and
// a calendar left behind after the account is gone is a frozen lie about a day.
func TestDisconnectingRemovesTheEvents(t *testing.T) {
g := newFakeCalendarGoogle(t)
g.events["fam"] = []map[string]any{
{"id": "e1", "summary": "Møde",
"start": map[string]string{"date": "2026-08-20"},
"end": map[string]string{"date": "2026-08-21"}},
}
ts, user := connectedCalendar(t, g, nil)
ts.do(t, "GET", "/api/v1/calendar", nil)
account, _ := ts.db.CalendarAccountFor(t.Context(), user.ID, "google")
calendars, _ := ts.db.Calendars(t.Context(), account.ID)
var famID string
for _, c := range calendars {
if c.RemoteID == "fam" {
famID = c.ID
}
}
ts.do(t, "PUT", "/api/v1/calendar/calendars", map[string]any{"shown": []string{famID}})
ts.do(t, "POST", "/api/v1/calendar/sync", nil)
if resp, _ := ts.do(t, "DELETE", "/api/v1/calendar", nil); resp.StatusCode != http.StatusNoContent {
t.Fatalf("disconnect: status %d", resp.StatusCode)
}
_, body := ts.do(t, "GET", "/api/v1/calendar/events?from=2026-08-01&to=2026-08-31", nil)
if events, _ := body["events"].([]any); len(events) != 0 {
t.Errorf("events survived the disconnect: %v", body["events"])
}
_, body = ts.do(t, "GET", "/api/v1/calendar", nil)
if body["connected"] != false {
t.Error("still reported as connected")
}
}
// A range that is not a date has to be refused, not guessed at. An empty `from`
// compared as a string matches everything before it, which is a grid quietly
// answering a question nobody asked.
func TestEventsRefuseARangeThatIsNotADate(t *testing.T) {
g := newFakeCalendarGoogle(t)
ts, _ := connectedCalendar(t, g, nil)
for _, query := range []string{
"", "?from=2026-08-01", "?from=&to=2026-08-31",
"?from=i+morgen&to=2026-08-31", "?from=2026-08-01&to=2026-13-40",
} {
resp, body := ts.do(t, "GET", "/api/v1/calendar/events"+query, nil)
if resp.StatusCode != http.StatusUnprocessableEntity {
t.Errorf("%q: status %d, want 422 (%v)", query, resp.StatusCode, body)
}
}
}
// A person with no calendar connected still opens the view. It has to answer, and
// answer emptily, rather than fail — the grid is also verdande's own tasks, and
// they must not disappear because Google is not set up.
func TestTheGridAnswersWithNoCalendarConnected(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
resp, body := ts.do(t, "GET", "/api/v1/calendar/events?from=2026-08-01&to=2026-08-31", nil)
if resp.StatusCode != http.StatusOK {
t.Fatalf("status %d, body %v", resp.StatusCode, body)
}
if events, ok := body["events"].([]any); !ok || len(events) != 0 {
t.Errorf("events = %v, want an empty list rather than null", body["events"])
}
resp, body = ts.do(t, "GET", "/api/v1/calendar", nil)
if resp.StatusCode != http.StatusOK || body["connected"] != false {
t.Fatalf("status %d, body %v", resp.StatusCode, body)
}
if body["has_client"] != false {
t.Error("an instance with no OAuth registration claimed to have one; the page " +
"would offer a button that answers 409")
}
// Spelled out even when nothing is connected: it is what has to be registered
// with Google, and it is the single most likely thing to be wrong.
if uri, _ := body["redirect_uri"].(string); !strings.HasSuffix(uri, "/oauth/calendar/callback") {
t.Errorf("redirect_uri = %q", uri)
}
}
// A slow Google must not hold the request open until something in front of this
// server gives up on it and answers with its own HTML.
func TestASlowCalendarDoesNotHangTheRequest(t *testing.T) {
blocked := make(chan struct{})
defer close(blocked)
slow := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
select {
case <-blocked:
case <-r.Context().Done():
}
}))
defer slow.Close()
ts := newTestServerWith(t, func(cfg *config.Config) {
cfg.GmailClientID = "test-client"
cfg.GmailClientSecret = "test-secret"
cfg.GoogleTokenURL = slow.URL + "/token"
cfg.CalendarAPIURL = slow.URL
// A second rather than the real twenty-five: what is being tested is that a
// budget is applied at all, and spending the production one on every run
// would put half a minute into the suite to learn the same thing.
cfg.CalendarSyncBudget = time.Second
})
ts.bootstrap(t)
user, err := ts.db.UserByEmail(t.Context(), "[email protected]")
if err != nil {
t.Fatal(err)
}
if err := ts.db.SaveCalendarAccount(t.Context(), &store.CalendarAccount{
UserID: user.ID, Provider: "google", RefreshToken: "r",
AccessToken: "a", ExpiresAt: time.Now().Add(time.Hour),
}); err != nil {
t.Fatal(err)
}
// Its own client, because the shared one gives up after ten seconds — which is
// shorter than the budget being tested and would prove nothing either way.
req, _ := http.NewRequest("POST", ts.URL+"/api/v1/calendar/sync", nil)
req.Header.Set("Sec-Fetch-Site", "same-origin")
for _, c := range ts.client.Jar.Cookies(mustParse(t, ts.URL)) {
req.AddCookie(c)
}
started := time.Now()
resp, err := (&http.Client{Timeout: 90 * time.Second}).Do(req)
if err != nil {
t.Fatalf("the request never came back (%v after %s); the sync has no budget of its "+
"own, and whatever sits in front of this server answers for it",
err, time.Since(started).Round(time.Second))
}
defer resp.Body.Close()
if took := time.Since(started); took > 10*time.Second {
t.Errorf("the sync took %s; the budget is not being applied", took.Round(time.Second))
}
if resp.StatusCode >= 500 {
t.Errorf("status %d — a slow Google is not this server breaking", resp.StatusCode)
}
}
+84 -9
View File
@@ -225,17 +225,32 @@ func (s *Server) handleUploadAttachment(w http.ResponseWriter, r *http.Request)
a := &store.Attachment{
TaskID: taskID,
CommentID: r.FormValue("comment_id"),
Filename: safeFilename(header.Filename),
MimeType: detectMime(header.Filename, header.Header.Get("Content-Type")),
Size: size,
Path: stored,
UploadedBy: user.ID,
}
// An attachment hangs on a task or on a comment, never both — the schema says
// so, and the form may have supplied a comment id.
if a.CommentID != "" {
a.TaskID = ""
//
// The comment is resolved and checked against the task in the URL. It was
// taken on trust, which meant the permission check above guarded the *task
// somebody named* while the file landed on *whatever comment they asked for*.
// A viewer on a shared project can list its comments, so she could read an id
// there and then post to a task in her own project with that id attached: the
// file appears under somebody else's comment, in a project she may only read,
// and is served to everyone in it from this origin under a name she chose.
// Somebody removed from a project could do the same with ids they kept.
if id := r.FormValue("comment_id"); id != "" {
owner, err := s.db.CommentTask(r.Context(), id)
if err != nil || owner != taskID {
os.Remove(filepath.Join(s.cfg.FilesDir(), stored))
writeError(w, http.StatusNotFound, CodeNotFound, "not found")
return
}
a.CommentID, a.TaskID = id, ""
}
if err := s.db.CreateAttachment(r.Context(), a); err != nil {
@@ -327,6 +342,30 @@ func (s *Server) reachAttachment(r *http.Request, attachmentID string) (canEdit
return true, nil
}
// A file inside a note follows the note, which follows its project — the same
// chain everything else here follows. Without this a picture imported with a
// note answered 404 while sitting in the middle of the text that named it.
if a.NoteID != "" {
n, err := s.db.Note(r.Context(), a.NoteID)
if err != nil {
return false, err
}
if n == nil {
return false, store.ErrNotFound
}
if n.ProjectID == "" {
if n.CreatedBy != user.ID {
return false, store.ErrNotFound
}
return true, nil
}
role, err := store.RequireProjectRole(r.Context(), s.db, n.ProjectID, user.ID, store.RoleViewer)
if err != nil {
return false, err
}
return role.CanEdit(), nil
}
taskID, err := s.db.AttachmentTask(r.Context(), attachmentID)
if err != nil {
return false, err
@@ -370,12 +409,30 @@ func (s *Server) handleDownloadAttachment(w http.ResponseWriter, r *http.Request
}
defer f.Close()
// Everything is served as a download rather than rendered. An uploaded SVG or
// HTML file displayed inline would run its own script on this origin, with the
// session cookie attached — attachments are the one place a user supplies
// content that another user opens.
w.Header().Set("Content-Type", "application/octet-stream")
w.Header().Set("Content-Disposition", "attachment; filename*=UTF-8''"+
// Almost everything is served as a download rather than rendered. An uploaded
// SVG or HTML file displayed inline would run its own script on this origin,
// with the session cookie attached — attachments are the one place a user
// supplies content that another user opens.
//
// The exception is a short list of raster images, and it is not a softening of
// that rule but what makes a note work: a note carries its pictures in the text
// as `![](…)`, and served as octet-stream every one of them is a broken image.
// The whole Apple Notes import exists to bring those pictures along.
//
// Safe because it is an allowlist of formats that cannot carry script, sent as
// the exact type with `nosniff` — so the browser must treat it as that image or
// not at all, and can never be talked into reading it as HTML. SVG is
// deliberately absent: it is a document, not a picture, and it is the one image
// type that can run script.
disposition := "attachment"
contentType := "application/octet-stream"
if inlineImage[a.MimeType] {
disposition = "inline"
contentType = a.MimeType
}
w.Header().Set("Content-Type", contentType)
w.Header().Set("Content-Disposition", disposition+"; filename*=UTF-8''"+
strings.ReplaceAll(mime.QEncoding.Encode("utf-8", a.Filename), " ", "%20"))
w.Header().Set("X-Content-Type-Options", "nosniff")
w.Header().Set("Content-Length", fmt.Sprint(a.Size))
@@ -384,6 +441,24 @@ func (s *Server) handleDownloadAttachment(w http.ResponseWriter, r *http.Request
http.ServeContent(w, r, a.Filename, a.CreatedAt, f)
}
// inlineImage is what may be shown rather than downloaded.
//
// Raster formats only, and named one by one rather than matched on an "image/"
// prefix: the prefix would let image/svg+xml through, which is a document that
// can run script, and a list that grows by accident is how that happens.
var inlineImage = map[string]bool{
"image/png": true,
"image/jpeg": true,
"image/gif": true,
"image/webp": true,
"image/avif": true,
// Sendes med, selv om ingen browser viser den endnu. Den er lige så ufarlig
// som de andre, og en iPhone-note er fuld af dem — den dag Safari viser dem,
// virker de, uden at nogen skal huske hvorfor de ikke gjorde.
"image/heic": true,
"image/heif": true,
}
func (s *Server) handleDeleteAttachment(w http.ResponseWriter, r *http.Request) {
attachmentID := chi.URLParam(r, "attachmentID")
+66
View File
@@ -0,0 +1,66 @@
package httpapi
import (
"strings"
"testing"
)
// Politikken skal stå i svaret, og den skal stadig være stram.
//
// Den er dét, der gjorde en XSS i noteeditoren til indsat HTML frem for kørt
// script, dengang en `alt` slap uden om sin undslupning — og der var ingen prøve
// på, at headeren overhovedet blev sendt. En sikkerhedsforanstaltning, ingen
// prøve rører, er en, der kan forsvinde i en oprydning uden at nogen opdager det.
func TestEveryResponseCarriesAStrictCSP(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
resp, _ := ts.do(t, "GET", "/api/v1/auth/me", nil)
policy := resp.Header.Get("Content-Security-Policy")
if policy == "" {
t.Fatal("der er ingen Content-Security-Policy på svaret")
}
// Målt på script-src alene, og ikke på hele politikken.
//
// `style-src 'unsafe-inline'` står der med vilje og er forklaret i csp.go:
// Svelte skriver stilarter ind i dokumentet, og en indsat stil kan gøre en
// side grim, ikke køre kode. Det er script-src, der afgør, om indsat HTML
// bliver til et kørende script, og det er derfor den, prøven ser på.
scriptSrc := ""
for _, part := range strings.Split(policy, ";") {
if strings.HasPrefix(strings.TrimSpace(part), "script-src") {
scriptSrc = part
}
}
if scriptSrc == "" {
t.Fatal("der er ingen script-src i politikken")
}
for _, forbidden := range []string{"'unsafe-inline'", "'unsafe-eval'", "'unsafe-hashes'", "'strict-dynamic'"} {
if strings.Contains(scriptSrc, forbidden) {
t.Errorf("script-src indeholder %s: %q", forbidden, strings.TrimSpace(scriptSrc))
}
}
// Og de fire, der skal være der.
for _, want := range []string{"script-src", "object-src 'none'", "base-uri 'none'", "frame-ancestors 'none'"} {
if !strings.Contains(policy, want) {
t.Errorf("politikken mangler %q", want)
}
}
}
// Et bilag må ikke kunne blive læst som et dokument.
func TestAttachmentsAreNotServedAsDocuments(t *testing.T) {
for _, mime := range []string{"image/svg+xml", "text/html", "application/xhtml+xml", "text/xml"} {
if inlineImage[mime] {
t.Errorf("%s står på listen over det, der vises inline", mime)
}
}
// Og listen skal faktisk kunne noget, ellers virker billeder i noter ikke.
for _, mime := range []string{"image/png", "image/jpeg"} {
if !inlineImage[mime] {
t.Errorf("%s mangler; billeder i noter kan ikke vises", mime)
}
}
}
+10
View File
@@ -58,6 +58,16 @@ const (
// read "something went wrong", and had nowhere to go. The interface wraps the
// message in one that says where it came from.
CodeGmailFailed = "gmail_failed"
// And the same for a mailbox read over IMAP. A host's refusal is the whole
// diagnosis — a wrong app password, a host that is not listening, a folder that
// is not there — and it says it in its own words.
CodeMailboxFailed = "mailbox_failed"
// And for a calendar read from Google. Its own code rather than the Gmail one,
// because the sentence the interface wraps it in names where it came from — and
// "Gmail said no" about a calendar sends somebody to the wrong panel.
CodeCalendarFailed = "calendar_failed"
)
// errLastAdmin is returned to the caller that already wrote the response, so it
+92 -55
View File
@@ -25,18 +25,16 @@ const defaultGmailSyncBudget = 25 * time.Second
// to redeploy a container to paste a client id.
// gmailClient is the API client, pointed wherever the config says Google is.
func (s *Server) gmailClient(accessToken string) *gmail.Client {
return gmail.NewClient(accessToken).At(gmail.Endpoints{
Token: s.cfg.GmailTokenURL, API: s.cfg.GmailAPIURL,
})
return gmail.NewClient(accessToken).At(s.cfg.GmailAPIURL)
}
func (s *Server) gmailConfig(ctx context.Context) gmail.Config {
endpoints := gmail.Endpoints{Token: s.cfg.GmailTokenURL, API: s.cfg.GmailAPIURL}
cfg := gmail.Config{
ClientID: s.cfg.GmailClientID,
ClientSecret: s.cfg.GmailClientSecret,
RedirectURL: s.cfg.GmailRedirectURL(),
Endpoints: endpoints,
Scope: gmail.Scope,
TokenURL: s.cfg.GoogleTokenURL,
}
if cfg.ClientID != "" && cfg.ClientSecret != "" {
return cfg
@@ -179,15 +177,27 @@ func (s *Server) handleGmailCallback(w http.ResponseWriter, r *http.Request) {
delete(settings, "pkce_state")
delete(settings, "pkce_expires")
settings["refresh_token"] = token.RefreshToken
settings["access_token"] = token.AccessToken
settings["expires_at"] = token.ExpiresAt.Unix()
settings["email"] = email
if _, ok := settings["trigger"]; !ok {
settings["trigger"] = "starred"
if err := s.db.SetUserSettings(r.Context(), user.ID, "gmail", settings); err != nil {
s.internal(w, r, "clear gmail handshake", err)
return
}
if err := s.db.SetUserSettings(r.Context(), user.ID, "gmail", settings); err != nil {
// The mailbox itself, kept beside the ones read over IMAP. Connecting again
// writes the same row: Gmail is one per person, because the OAuth flow is.
box, err := s.db.MailboxOfKind(r.Context(), user.ID, "gmail")
if err != nil {
s.internal(w, r, "read gmail mailbox", err)
return
}
if box == nil {
box = &store.Mailbox{UserID: user.ID, Kind: "gmail", Trigger: "starred"}
}
box.Name = email
box.Username = email
box.RefreshToken = token.RefreshToken
box.AccessToken = token.AccessToken
box.ExpiresAt = token.ExpiresAt
if err := s.db.SaveMailbox(r.Context(), box); err != nil {
s.internal(w, r, "save gmail tokens", err)
return
}
@@ -239,37 +249,52 @@ func (s *Server) handleGmailSyncNow(w http.ResponseWriter, r *http.Request) {
//
// Returns how many were made. Also used by the background job, which is why it
// takes a context and a user rather than a request.
// maxGmailSeen bounds the list of message ids already turned into tasks.
//
// Gmail is read-only here — the scope is `gmail.readonly`, deliberately, because
// asking for less is what makes the consent screen something a person can agree
// to — so verdande cannot unstar a message the way the IMAP side unflags one. The
// record has to live on this side, and a record on this side has to be bounded.
//
// A count rather than the previous rule of "whatever Gmail returned this run",
// which was not a bound but a way of forgetting.
const maxGmailSeen = 2000
func (s *Server) SyncGmail(ctx context.Context, user *store.User) (int, error) {
settings, err := s.db.UserSettings(ctx, user.ID, "gmail")
box, err := s.db.MailboxOfKind(ctx, user.ID, "gmail")
if err != nil {
return 0, err
}
str := func(key string) string {
v, _ := settings[key].(string)
return v
if box != nil {
// The same guard as the mailboxes read over IMAP, for the same reason: a
// press of "Fetch now" during the sweep would otherwise make two runs from
// one marker.
unlock := s.lockSync(box.ID)
defer unlock()
if fresh, err := s.db.MailboxOfKind(ctx, user.ID, "gmail"); err == nil && fresh != nil {
box = fresh
}
}
refresh := str("refresh_token")
if refresh == "" {
if box == nil || box.RefreshToken == "" {
return 0, nil // not connected; nothing to do and nothing to complain about
}
query := gmail.Query(str("trigger"), str("label"))
query := gmail.Query(box.Trigger, box.Label)
if query == "" {
return 0, nil // connected but no trigger chosen
}
refresh := box.RefreshToken
// The access token is refreshed only when it has actually expired.
access := str("access_token")
expires, _ := settings["expires_at"].(float64)
if access == "" || int64(expires) <= time.Now().Unix() {
access := box.AccessToken
if access == "" || box.ExpiresAt.Before(time.Now()) {
token, err := s.gmailConfig(ctx).Refresh(ctx, refresh)
if err != nil {
return 0, err
}
access = token.AccessToken
settings["access_token"] = access
settings["expires_at"] = token.ExpiresAt.Unix()
if err := s.db.SetUserSettings(ctx, user.ID, "gmail", settings); err != nil {
box.AccessToken = access
box.ExpiresAt = token.ExpiresAt
if err := s.db.SaveMailbox(ctx, box); err != nil {
return 0, err
}
}
@@ -281,7 +306,7 @@ func (s *Server) SyncGmail(ctx context.Context, user *store.User) (int, error) {
// The person revoked access, or changed their password. Forget the
// tokens rather than retrying every ten minutes forever.
s.log.Info("gmail access was revoked; disconnecting", "user", user.ID)
_ = s.db.SetUserSettings(ctx, user.ID, "gmail", map[string]any{})
_ = s.db.DeleteMailbox(ctx, user.ID, box.ID)
}
return 0, err
}
@@ -290,19 +315,15 @@ func (s *Server) SyncGmail(ctx context.Context, user *store.User) (int, error) {
// rather than as a table: it is bounded by the query's own 30-day window, and
// a table would be one more thing to migrate for a feature this small.
seen := map[string]bool{}
if raw, ok := settings["seen"].([]any); ok {
for _, v := range raw {
if id, ok := v.(string); ok {
seen[id] = true
}
}
for _, id := range box.Seen {
seen[id] = true
}
inbox, err := s.db.InboxID(ctx, user.ID)
if err != nil {
return 0, err
}
projectID := str("project_id")
projectID := box.ProjectID
if projectID == "" {
projectID = inbox
} else if _, err := store.RequireProjectRole(ctx, s.db, projectID, user.ID, store.RoleEditor); err != nil {
@@ -350,17 +371,38 @@ func (s *Server) SyncGmail(ctx context.Context, user *store.User) (int, error) {
}
if created > 0 {
// Only the ids Gmail still returns are kept, so the list cannot grow
// without bound as messages fall out of the 30-day window.
keep := make([]any, 0, len(ids))
// The ids this run saw, on top of the ones already known.
//
// It used to keep only the ids Gmail returned *this run*, to stop the list
// growing without bound. That threw away every id that was not on the page
// it happened to fetch — and the query is capped, so on an account with more
// starred mail than one page, each run forgot the messages the last one had
// remembered. They came back unseen, and became tasks again, and again. That
// is the duplicate.
//
// Bounded by a count instead. Newest first, and a ceiling far above any
// plausible page, so an id leaves the list because it is old rather than
// because it was not in the last answer.
keep := make([]string, 0, len(ids)+len(box.Seen))
added := map[string]bool{}
for _, id := range ids {
if seen[id] {
if seen[id] && !added[id] {
keep = append(keep, id)
added[id] = true
}
}
settings["seen"] = keep
settings["last_sync"] = time.Now().Unix()
if err := s.db.SetUserSettings(ctx, user.ID, "gmail", settings); err != nil {
for _, id := range box.Seen {
if !added[id] {
keep = append(keep, id)
added[id] = true
}
}
if len(keep) > maxGmailSeen {
keep = keep[:maxGmailSeen]
}
box.Seen = keep
box.LastSyncAt = time.Now()
if err := s.db.SaveMailbox(ctx, box); err != nil {
return created, err
}
}
@@ -483,30 +525,25 @@ func (s *Server) handleSetGmailClient(w http.ResponseWriter, r *http.Request) {
// natural place to learn the address, and it is also the one moment that cannot be
// retried — so the status endpoint asks again when it finds nothing, and the answer
// corrects itself on the next look.
func (s *Server) gmailProfile(ctx context.Context, user *store.User, settings map[string]any) (string, error) {
str := func(key string) string {
v, _ := settings[key].(string)
return v
}
access := str("access_token")
expires, _ := settings["expires_at"].(float64)
if access == "" || int64(expires) <= time.Now().Unix() {
token, err := s.gmailConfig(ctx).Refresh(ctx, str("refresh_token"))
func (s *Server) gmailProfile(ctx context.Context, box *store.Mailbox) (string, error) {
access := box.AccessToken
if access == "" || box.ExpiresAt.Before(time.Now()) {
token, err := s.gmailConfig(ctx).Refresh(ctx, box.RefreshToken)
if err != nil {
return "", err
}
access = token.AccessToken
settings["access_token"] = access
settings["expires_at"] = token.ExpiresAt.Unix()
box.AccessToken = access
box.ExpiresAt = token.ExpiresAt
}
email, err := s.gmailClient(access).Profile(ctx)
if err != nil {
return "", err
}
settings["email"] = email
if err := s.db.SetUserSettings(ctx, user.ID, "gmail", settings); err != nil {
box.Name = email
box.Username = email
if err := s.db.SaveMailbox(ctx, box); err != nil {
return "", err
}
return email, nil
+8 -7
View File
@@ -1,6 +1,7 @@
package httpapi
import (
"github.com/kristianwind/verdande/internal/store"
"net/http"
"net/http/httptest"
"net/url"
@@ -28,7 +29,7 @@ func TestAGmailFailureSaysWhatGmailSaidAndIsRecorded(t *testing.T) {
ts := newTestServerWith(t, func(cfg *config.Config) {
cfg.GmailClientID = "test-client"
cfg.GmailClientSecret = "test-secret"
cfg.GmailTokenURL = google.URL + "/token"
cfg.GoogleTokenURL = google.URL + "/token"
cfg.GmailAPIURL = google.URL
})
ts.bootstrap(t)
@@ -38,8 +39,8 @@ func TestAGmailFailureSaysWhatGmailSaidAndIsRecorded(t *testing.T) {
t.Fatal(err)
}
// Connected, with a refresh token that will be refused.
if err := ts.db.SetUserSettings(t.Context(), user.ID, "gmail", map[string]any{
"refresh_token": "spent", "trigger": "starred",
if err := ts.db.SaveMailbox(t.Context(), &store.Mailbox{
UserID: user.ID, Kind: "gmail", RefreshToken: "spent", Trigger: "starred",
}); err != nil {
t.Fatal(err)
}
@@ -96,7 +97,7 @@ func TestASlowMailboxDoesNotHangTheRequest(t *testing.T) {
ts := newTestServerWith(t, func(cfg *config.Config) {
cfg.GmailClientID = "test-client"
cfg.GmailClientSecret = "test-secret"
cfg.GmailTokenURL = google.URL + "/token"
cfg.GoogleTokenURL = google.URL + "/token"
cfg.GmailAPIURL = google.URL
// A second rather than the real twenty-five: what is being tested is that a
// budget is applied at all, and spending the production one on every run
@@ -111,9 +112,9 @@ func TestASlowMailboxDoesNotHangTheRequest(t *testing.T) {
}
// A token that has not expired, so the run goes straight to the message list
// rather than stopping at a refresh.
if err := ts.db.SetUserSettings(t.Context(), user.ID, "gmail", map[string]any{
"refresh_token": "r", "access_token": "a", "trigger": "starred",
"expires_at": time.Now().Add(time.Hour).Unix(),
if err := ts.db.SaveMailbox(t.Context(), &store.Mailbox{
UserID: user.ID, Kind: "gmail", RefreshToken: "r", AccessToken: "a",
Trigger: "starred", ExpiresAt: time.Now().Add(time.Hour),
}); err != nil {
t.Fatal(err)
}
+61 -26
View File
@@ -11,6 +11,7 @@ import (
"github.com/kristianwind/verdande/internal/ai"
"github.com/kristianwind/verdande/internal/auth"
"github.com/kristianwind/verdande/internal/quickadd"
"github.com/kristianwind/verdande/internal/safedial"
"github.com/kristianwind/verdande/internal/store"
)
@@ -214,6 +215,24 @@ func (s *Server) handleSetAISettings(w http.ResponseWriter, r *http.Request) {
return
}
// The address is checked where it is typed, as well as when it is dialled.
//
// safedial refuses a private destination at connect time and is the actual
// wall; this is so the person entering it hears about it now rather than
// discovering it later as a failed summary. A scheme other than http or https
// is refused outright — file:// and gopher:// are not providers.
if url := strings.TrimSpace(req.BaseURL); url != "" {
if !strings.HasPrefix(url, "http://") && !strings.HasPrefix(url, "https://") {
writeFieldErrors(w, map[string]string{"base_url": "must start with http:// or https://"})
return
}
if reason := safedial.CheckURL(url); reason != "" {
writeFieldErrors(w, map[string]string{"base_url": reason})
return
}
req.BaseURL = url
}
existing, err := s.aiConfig(r, user.ID)
if err != nil {
s.internal(w, r, "ai settings", err)
@@ -370,27 +389,28 @@ type gmailSettings struct {
}
func (s *Server) handleGetGmail(w http.ResponseWriter, r *http.Request) {
values, err := s.db.UserSettings(r.Context(), userFrom(r.Context()).ID, "gmail")
user := userFrom(r.Context())
box, err := s.db.MailboxOfKind(r.Context(), user.ID, "gmail")
if err != nil {
s.internal(w, r, "gmail settings", err)
s.internal(w, r, "gmail mailbox", err)
return
}
str := func(key string) string {
v, _ := values[key].(string)
return v
if box == nil {
writeJSON(w, http.StatusOK, gmailSettings{})
return
}
// The mailbox address, fetched again if it is missing.
//
// It is stored at connect time, and the call that fetches it can fail — a
// slow token, a scope Google had not applied yet. That failure used to be a
// log line and an empty string, so the page said "an unknown account" and went
// on saying it forever: the only way back was to disconnect and reconnect,
// which nobody would guess. Asking again here costs one request on a page
// nobody opens often, and it means the wrong answer lasts until the next look
// rather than until somebody reconnects.
email := str("email")
if email == "" && str("refresh_token") != "" {
if fetched, err := s.gmailProfile(r.Context(), userFrom(r.Context()), values); err == nil {
// It is stored at connect time, and the call that fetches it can fail — a slow
// token, a scope Google had not applied yet. That failure used to be a log line
// and an empty string, so the page said "an unknown account" and went on saying
// it forever: the only way back was to disconnect and reconnect, which nobody
// would guess. Asking again here costs one request on a page nobody opens
// often, and it means the wrong answer lasts until the next look.
email := box.Username
if email == "" && box.RefreshToken != "" {
if fetched, err := s.gmailProfile(r.Context(), box); err == nil {
email = fetched
} else {
s.log.Warn("gmail profile", "err", err)
@@ -398,10 +418,10 @@ func (s *Server) handleGetGmail(w http.ResponseWriter, r *http.Request) {
}
writeJSON(w, http.StatusOK, gmailSettings{
Connected: str("refresh_token") != "",
Connected: box.RefreshToken != "",
Email: email,
Trigger: str("trigger"),
Label: str("label"),
Trigger: box.Trigger,
Label: box.Label,
})
}
@@ -412,27 +432,42 @@ func (s *Server) handleSetGmail(w http.ResponseWriter, r *http.Request) {
}
user := userFrom(r.Context())
values, err := s.db.UserSettings(r.Context(), user.ID, "gmail")
box, err := s.db.MailboxOfKind(r.Context(), user.ID, "gmail")
if err != nil {
s.internal(w, r, "gmail settings", err)
s.internal(w, r, "gmail mailbox", err)
return
}
values["trigger"] = req.Trigger
values["label"] = req.Label
if box == nil {
// Choosing what makes a task before connecting is allowed, and the choice is
// kept: the row is the mailbox with no credential in it yet, which reads as
// not connected everywhere that asks and is skipped by the sweep.
box = &store.Mailbox{UserID: user.ID, Kind: "gmail"}
}
box.Trigger = req.Trigger
box.Label = req.Label
if err := s.db.SetUserSettings(r.Context(), user.ID, "gmail", values); err != nil {
if err := s.db.SaveMailbox(r.Context(), box); err != nil {
s.internal(w, r, "save gmail settings", err)
return
}
w.WriteHeader(http.StatusNoContent)
}
// handleDisconnectGmail forgets the connection, tokens and all.
// handleDisconnectGmail forgets the connection, tokens and all. The tasks it made
// stay: they are the person's work now, not the mailbox's.
func (s *Server) handleDisconnectGmail(w http.ResponseWriter, r *http.Request) {
if err := s.db.SetUserSettings(r.Context(), userFrom(r.Context()).ID, "gmail", map[string]any{}); err != nil {
s.internal(w, r, "disconnect gmail", err)
user := userFrom(r.Context())
box, err := s.db.MailboxOfKind(r.Context(), user.ID, "gmail")
if err != nil {
s.internal(w, r, "gmail mailbox", err)
return
}
if box != nil {
if err := s.db.DeleteMailbox(r.Context(), user.ID, box.ID); err != nil {
s.internal(w, r, "disconnect gmail", err)
return
}
}
w.WriteHeader(http.StatusNoContent)
}
+265
View File
@@ -0,0 +1,265 @@
package httpapi
import (
"context"
"errors"
"fmt"
"net/http"
"strings"
"sync"
"time"
"github.com/go-chi/chi/v5"
"github.com/kristianwind/verdande/internal/imap"
"github.com/kristianwind/verdande/internal/store"
)
// defaultMailboxSyncBudget bounds one read of one mailbox when somebody is waiting.
// The same reasoning as Gmail's: a mailbox that has stopped answering must not
// hold a request open until whatever sits in front of this server answers for it.
const defaultMailboxSyncBudget = 25 * time.Second
func (s *Server) handleListMailboxes(w http.ResponseWriter, r *http.Request) {
user := userFrom(r.Context())
list, err := s.db.Mailboxes(r.Context(), user.ID)
if err != nil {
s.internal(w, r, "list mailboxes", err)
return
}
if list == nil {
list = []store.Mailbox{}
}
writeJSON(w, http.StatusOK, map[string]any{"mailboxes": list})
}
// handleAddMailbox connects one, after proving it can actually be read.
//
// Tested before it is saved, not after: a mailbox that was accepted and then
// silently fails every ten minutes is worse than one that refused at the door,
// where the person is still holding the password they just typed.
func (s *Server) handleAddMailbox(w http.ResponseWriter, r *http.Request) {
var body struct {
Name string `json:"name"`
Host string `json:"host"`
Username string `json:"username"`
Password string `json:"password"`
Folder string `json:"folder"`
}
if err := decodeJSON(w, r, &body); err != nil {
return
}
body.Host = strings.TrimSpace(body.Host)
body.Username = strings.TrimSpace(body.Username)
if body.Host == "" || body.Username == "" || body.Password == "" {
writeError(w, http.StatusBadRequest, CodeValidation,
"a mailbox needs a host, a username and a password")
return
}
// A host with no port is the mistake everyone makes once, and 993 is the answer
// for every host worth naming.
if !strings.Contains(body.Host, ":") {
body.Host += ":993"
}
account := imap.Account{
Host: body.Host, Username: body.Username,
Password: body.Password, Folder: body.Folder,
}
client, err := imap.Dial(account)
if err != nil {
s.upstream(w, r, CodeMailboxFailed, "connect mailbox", err)
return
}
client.Close()
user := userFrom(r.Context())
m := &store.Mailbox{
UserID: user.ID, Kind: "imap", Name: strings.TrimSpace(body.Name),
Host: body.Host, Username: body.Username,
Password: body.Password, Folder: body.Folder,
}
if m.Name == "" {
m.Name = body.Username
}
if err := s.db.SaveMailbox(r.Context(), m); err != nil {
// The unique index. Saying which mailbox is already here beats "constraint
// failed", which is true and useless.
if strings.Contains(err.Error(), "UNIQUE") {
writeError(w, http.StatusConflict, CodeValidation,
fmt.Sprintf("%s on %s is already connected", body.Username, body.Host))
return
}
s.internal(w, r, "save mailbox", err)
return
}
writeJSON(w, http.StatusCreated, m)
}
func (s *Server) handleDeleteMailbox(w http.ResponseWriter, r *http.Request) {
user := userFrom(r.Context())
if err := s.db.DeleteMailbox(r.Context(), user.ID, chi.URLParam(r, "mailboxID")); err != nil {
s.internal(w, r, "delete mailbox", err)
return
}
w.WriteHeader(http.StatusNoContent)
}
// handleSyncMailbox reads one mailbox now, within a budget of its own.
func (s *Server) handleSyncMailbox(w http.ResponseWriter, r *http.Request) {
user := userFrom(r.Context())
m, err := s.db.Mailbox(r.Context(), user.ID, chi.URLParam(r, "mailboxID"))
if err != nil {
s.internal(w, r, "read mailbox", err)
return
}
if m == nil {
writeError(w, http.StatusNotFound, CodeNotFound, "no such mailbox")
return
}
ctx, cancel := context.WithTimeout(r.Context(), defaultMailboxSyncBudget)
defer cancel()
created, err := s.SyncMailbox(ctx, user, m)
if err != nil {
// Whatever was made by the deadline is kept: each task is committed as it
// is made, so a short budget is a shorter run rather than a lost one.
if errors.Is(err, context.DeadlineExceeded) {
writeJSON(w, http.StatusOK, map[string]any{"created": created, "partial": true})
return
}
s.upstream(w, r, CodeMailboxFailed, "sync mailbox", err)
return
}
writeJSON(w, http.StatusOK, map[string]any{"created": created})
}
// SyncMailbox turns the flagged mail newer than the last run into tasks.
//
// Exported because the background job calls it too, on the same terms and without
// a budget: nobody is waiting there.
func (s *Server) SyncMailbox(ctx context.Context, user *store.User, m *store.Mailbox) (int, error) {
// One reader of this mailbox at a time. Held for the whole run, marker and all:
// releasing before the marker is written would leave exactly the gap this is
// here to close.
unlock := s.lockSync(m.ID)
defer unlock()
// Re-read under the lock. The row that came in may have been fetched before the
// run that just finished moved the marker.
if fresh, err := s.db.Mailbox(ctx, m.UserID, m.ID); err == nil && fresh != nil {
m = fresh
}
client, err := imap.Dial(imap.Account{
Host: m.Host, Username: m.Username, Password: m.Password, Folder: m.Folder,
})
if err != nil {
return 0, err
}
defer client.Close()
messages, searched, err := client.Since(m.LastUID, 25)
if err != nil {
return 0, err
}
if len(messages) == 0 {
return 0, nil
}
projectID, err := s.db.InboxID(ctx, user.ID)
if err != nil {
return 0, err
}
created := 0
highest := m.LastUID
// The uids of the mails that actually became tasks. Only those get unflagged,
// and only after the task is committed: a flag removed from a mail whose task
// was never written is a mail nobody will ever see again.
done := make([]uint32, 0, len(messages))
for _, msg := range messages {
if ctx.Err() != nil {
// What was made so far is already committed and counted, and the marker
// below has to be written for it: a run that created tasks and forgot how
// far it got would make them all again on the next pass.
break
}
subject := msg.Subject
if subject == "" {
subject = "(uden emne)"
}
content := subject
if msg.From != "" {
content = msg.From + ": " + subject
}
task := &store.Task{
ProjectID: projectID,
Content: content,
Description: msg.Snippet,
Priority: 4,
CreatedBy: user.ID,
}
if err := s.db.CreateTask(ctx, task, nil); err != nil {
s.log.Warn("mailbox create task", "err", err, "mailbox", m.ID)
continue
}
created++
done = append(done, msg.UID)
if msg.UID > highest {
highest = msg.UID
}
s.hub.Publish(projectID, "task.created", toTaskJSON(*task))
}
// The flag comes off what became a task. This is what actually stops a mail
// arriving twice; the marker below is now a second line rather than the only
// one. See imap.Unflag for why the record belongs in the mailbox and not here.
//
// Outside the request's context, like the marker: a cancelled request that
// already created tasks must still take the flags off them, or the next run
// makes them all again.
if err := client.Unflag(done); err != nil {
// Not fatal, and not the caller's problem. A folder somebody opened
// read-only, or a server that refuses the STORE, still gets the uid marker
// underneath — worse, but not broken.
s.log.Warn("could not unflag imported mail", "err", err, "mailbox", m.ID, "count", len(done))
}
// A run that got all the way through moves the marker to the whole of what the
// search considered, not merely to the last message it happened to read. That
// is what stops the same mail becoming a task again on the next run when the
// server did not put a uid in the fetch reply.
if ctx.Err() == nil && searched > highest {
highest = searched
}
// Written even when the loop was cut short, and outside the request's context
// so a cancelled request still records what it did.
if highest > m.LastUID {
if err := s.db.MarkMailboxRead(context.WithoutCancel(ctx), m.ID, highest, time.Now()); err != nil {
s.log.Warn("mark mailbox read", "err", err, "mailbox", m.ID)
}
}
if ctx.Err() != nil {
return created, ctx.Err()
}
return created, nil
}
// lockSync serialises one thing that polls against itself and returns the release.
//
// A mailbox to begin with; a calendar connection now takes the same lock under a
// key of its own. Two runs at once — somebody pressing "fetch now" while the sweep
// is in the middle of the same thing — both start from the state as it was, and
// the loser writes last. For a mailbox that made forty copies of one mail in a
// second; for a calendar it would be a window deleted after it was refilled.
func (s *Server) lockSync(id string) func() {
value, _ := s.syncing.LoadOrStore(id, &sync.Mutex{})
mu := value.(*sync.Mutex)
mu.Lock()
return mu.Unlock
}
+166
View File
@@ -101,6 +101,7 @@ func (s *Server) buildMCP() *mcp.Server {
}, "task_id", "body"),
}, s.mcpAddComment)
s.registerNoteTools(m)
return m
}
@@ -496,3 +497,168 @@ func clamp(value, fallback, max int) int {
}
return value
}
// --- notes ---------------------------------------------------------------------
// registerNoteTools adds the note half of the MCP surface.
//
// Separate from the task tools for one reason worth stating: this is how a note
// gets in from somewhere else. Apple Notes has no export worth the name, but a
// model running on the machine can read Notes.app and write what it finds through
// these — one note at a time, with the text as Markdown. That is a migration
// nobody has to build a parser for.
func (s *Server) registerNoteTools(m *mcp.Server) {
m.Register(mcp.Tool{
Name: "search_notes",
Description: "Find notes by text. Matches title and body, and is generous about " +
"Danish spelling — searching \"gron\" finds \"grøn\", and the other way round. " +
"With no query it returns the most recent notes. This is the tool for " +
"\"what did I write about the tax return\" or \"find my note on the kitchen\".",
InputSchema: mcp.Schema(map[string]any{
"text": mcp.Str("Words to look for in the title and body."),
"limit": mcp.Int("How many to return. Defaults to 25, maximum 100."),
}),
}, s.mcpSearchNotes)
m.Register(mcp.Tool{
Name: "create_note",
Description: "Write a note. The body is Markdown and is stored as typed — it is " +
"also exactly what an export produces, so nothing is converted and nothing " +
"is lost. Two things in the text are read out of it and become links: " +
"`#Projekt` files the note against that project, so it appears on the " +
"project's page without anybody filing it there, and `[[Another note]]` " +
"links to a note by its title. Use this to bring notes in from elsewhere — " +
"Apple Notes, OneNote, a folder of Markdown — one call per note.",
InputSchema: mcp.Schema(map[string]any{
"body": mcp.Str("The note as Markdown. Its first line becomes the title, so start with one. #Projekt and [[note]] become links."),
"project_id": mcp.Str("File it under a project by id. Usually unnecessary — a #tag in the body does the same and is what a person would type."),
"pinned": mcp.Bool("Keep it at the top of the list."),
}),
}, s.mcpCreateNote)
m.Register(mcp.Tool{
Name: "update_note",
Description: "Change a note. Only the fields given are changed. Sending `body` " +
"replaces the whole text, so read the note first if you mean to add to it — " +
"and the links are read again from the new text, so a #tag removed is a link " +
"removed.",
InputSchema: mcp.Schema(map[string]any{
"note_id": mcp.Str("Which note."),
"body": mcp.Str("The whole new text, as Markdown. The title follows its first line."),
"pinned": mcp.Bool("Pin it or unpin it."),
}),
}, s.mcpUpdateNote)
}
func (s *Server) mcpSearchNotes(ctx context.Context, userID string, args json.RawMessage) (any, error) {
var params struct {
Text string `json:"text"`
Limit int `json:"limit"`
}
_ = json.Unmarshal(args, &params)
limit := params.Limit
if limit <= 0 || limit > 100 {
limit = 25
}
var notes []store.Note
var err error
if strings.TrimSpace(params.Text) == "" {
notes, err = s.db.LooseNotes(ctx, userID)
} else {
notes, err = s.db.SearchNotes(ctx, params.Text, limit)
}
if err != nil {
return nil, err
}
// The same filter the HTTP surface applies, for the same reason: a search must
// not become a way to read what you cannot open.
out := make([]map[string]any, 0, len(notes))
for _, n := range notes {
if !s.mcpMayRead(ctx, &n, userID) {
continue
}
out = append(out, map[string]any{
"id": n.ID, "title": n.Title, "body": n.Body,
"project_id": n.ProjectID, "updated_at": n.UpdatedAt,
})
if len(out) >= limit {
break
}
}
return map[string]any{"notes": out, "count": len(out)}, nil
}
func (s *Server) mcpMayRead(ctx context.Context, n *store.Note, userID string) bool {
if n.ProjectID == "" {
return n.CreatedBy == userID
}
_, err := store.RequireProjectRole(ctx, s.db, n.ProjectID, userID, store.RoleViewer)
return err == nil
}
func (s *Server) mcpCreateNote(ctx context.Context, userID string, args json.RawMessage) (any, error) {
var params struct {
Body string `json:"body"`
ProjectID string `json:"project_id"`
Pinned bool `json:"pinned"`
}
_ = json.Unmarshal(args, &params)
n := &store.Note{
CreatedBy: userID,
Body: params.Body,
Pinned: params.Pinned,
}
if params.ProjectID != "" {
if _, err := store.RequireProjectRole(ctx, s.db, params.ProjectID, userID, store.RoleEditor); err != nil {
return nil, err
}
n.ProjectID = params.ProjectID
}
if err := s.db.SaveNote(ctx, n); err != nil {
return nil, err
}
// The links are reported back rather than left silent. A note whose #tag was
// misspelt looks identical to one whose tag landed, and in a chat nobody sees
// the row that was written.
return map[string]any{
"id": n.ID, "title": n.Title, "links": n.Links,
}, nil
}
func (s *Server) mcpUpdateNote(ctx context.Context, userID string, args json.RawMessage) (any, error) {
var params struct {
NoteID string `json:"note_id"`
Body *string `json:"body"`
Pinned *bool `json:"pinned"`
}
_ = json.Unmarshal(args, &params)
n, err := s.db.Note(ctx, params.NoteID)
if err != nil {
return nil, err
}
if n == nil || !s.mcpMayRead(ctx, n, userID) {
return nil, fmt.Errorf("no such note")
}
if n.ProjectID != "" {
if _, err := store.RequireProjectRole(ctx, s.db, n.ProjectID, userID, store.RoleEditor); err != nil {
return nil, err
}
}
if params.Body != nil {
n.Body = *params.Body
}
if params.Pinned != nil {
n.Pinned = *params.Pinned
}
if err := s.db.SaveNote(ctx, n); err != nil {
return nil, err
}
return map[string]any{"id": n.ID, "title": n.Title, "links": n.Links}, nil
}
+807
View File
@@ -0,0 +1,807 @@
package httpapi
import (
"archive/zip"
"fmt"
"io"
"mime"
"net/http"
"os"
"path"
"path/filepath"
"sort"
"strings"
"time"
"github.com/go-chi/chi/v5"
"github.com/kristianwind/verdande/internal/store"
)
// Notes.
//
// Access follows the project, because that is where a note lives: a note in a
// shared project is readable by everybody who can read the project and writable
// by an editor. A note with no project is its author's alone — the same answer a
// task in the inbox gets, arrived at the same way rather than by a rule of its own.
// mayReadNote reports whether this person may see the note, and mayWriteNote
// whether they may change it. Split, because a viewer on a shared project can do
// the first and not the second.
func (s *Server) mayTouchNote(r *http.Request, n *store.Note, role store.Role) bool {
user := userFrom(r.Context())
if n.ProjectID == "" {
return n.CreatedBy == user.ID
}
_, err := store.RequireProjectRole(r.Context(), s.db, n.ProjectID, user.ID, role)
return err == nil
}
func (s *Server) handleListNotes(w http.ResponseWriter, r *http.Request) {
user := userFrom(r.Context())
// Searching and listing are the same endpoint, because they answer the same
// question with different words. ?q= searches; ?project= narrows; neither gives
// the loose ones.
if q := strings.TrimSpace(r.URL.Query().Get("q")); q != "" {
found, err := s.db.SearchNotes(r.Context(), q, 50)
if err != nil {
s.internal(w, r, "search notes", err)
return
}
writeJSON(w, http.StatusOK, map[string]any{"notes": briefly(s.readable(r, found))})
return
}
if projectID := r.URL.Query().Get("project"); projectID != "" {
if _, err := store.RequireProjectRole(r.Context(), s.db, projectID, user.ID, store.RoleViewer); err != nil {
s.storeError(w, r, "notes in project", err)
return
}
found, err := s.db.NotesInProject(r.Context(), projectID)
if err != nil {
s.internal(w, r, "notes in project", err)
return
}
writeJSON(w, http.StatusOK, map[string]any{"notes": briefly(found)})
return
}
// The archive is the same list asked the other way round, so it is the same
// endpoint with a flag rather than a second one that would have to be kept in
// step with this.
if r.URL.Query().Get("archived") != "" {
found, err := s.db.ArchivedNotes(r.Context(), user.ID)
if err != nil {
s.internal(w, r, "list archived notes", err)
return
}
writeJSON(w, http.StatusOK, map[string]any{"notes": briefly(found)})
return
}
// Everything they can see, not only the loose ones. Filing a note in a project
// is how it gets shared, and a list that dropped it at that moment made sharing
// look like deleting.
found, err := s.db.AllNotes(r.Context(), user.ID)
if err != nil {
s.internal(w, r, "list notes", err)
return
}
writeJSON(w, http.StatusOK, map[string]any{"notes": briefly(found)})
}
// briefly replaces each note's body with the first of it.
//
// The list shows a title and about ninety characters. It was being sent whole
// notes to do that: with twelve hundred of them the response was 845 KB, of which
// ninety-eight per cent was body text nobody would read on that screen — fetched,
// parsed and held in memory on every visit.
//
// Body is emptied rather than shortened, and that is the important half. A field
// called `body` that holds a *piece* of the body is a trap: the editor would open
// it, the debounce would save it, and the note would be cut down to its own
// preview by nothing more than being looked at. Empty is a shape the client can
// check, and it does — see the notes page, which fetches the whole note when it
// opens one.
//
// Four hundred characters rather than ninety: the client strips Markdown before
// it shows anything, and a heading marker and a bold word are characters that
// vanish. Cheap insurance against a preview that ends up shorter than the line it
// is meant to fill.
const previewChars = 400
func briefly(notes []store.Note) []store.Note {
out := make([]store.Note, 0, len(notes))
for _, n := range notes {
body := []rune(n.Body)
if len(body) > previewChars {
body = body[:previewChars]
}
n.Preview = string(body)
n.Body = ""
out = append(out, n)
}
return out
}
// readable drops what this person may not see.
//
// Filtered after the search rather than inside it: FTS5 answers about text, and
// teaching the index about roles would put an access rule in a place nobody would
// think to look when it was wrong.
func (s *Server) readable(r *http.Request, notes []store.Note) []store.Note {
out := make([]store.Note, 0, len(notes))
for _, n := range notes {
if s.mayTouchNote(r, &n, store.RoleViewer) {
out = append(out, n)
}
}
return out
}
func (s *Server) handleGetNote(w http.ResponseWriter, r *http.Request) {
n, err := s.db.Note(r.Context(), chi.URLParam(r, "noteID"))
if err != nil {
s.internal(w, r, "get note", err)
return
}
// The same answer for "no such note" and "not yours", so an id cannot be probed.
if n == nil || !s.mayTouchNote(r, n, store.RoleViewer) {
writeError(w, http.StatusNotFound, CodeNotFound, "no such note")
return
}
writeJSON(w, http.StatusOK, n)
}
// No title. It is the note's first line and is derived on every save — a field
// here would be a promise this server cannot keep.
type noteBody struct {
ProjectID *string `json:"project_id"`
Body *string `json:"body"`
Pinned *bool `json:"pinned"`
}
func (s *Server) handleCreateNote(w http.ResponseWriter, r *http.Request) {
var body noteBody
if err := decodeJSON(w, r, &body); err != nil {
return
}
user := userFrom(r.Context())
n := &store.Note{CreatedBy: user.ID}
if body.ProjectID != nil && *body.ProjectID != "" {
if _, err := store.RequireProjectRole(r.Context(), s.db, *body.ProjectID, user.ID, store.RoleEditor); err != nil {
s.storeError(w, r, "create note", err)
return
}
n.ProjectID = *body.ProjectID
}
if body.Body != nil {
n.Body = *body.Body
}
if body.Pinned != nil {
n.Pinned = *body.Pinned
}
if err := s.db.SaveNote(r.Context(), n); err != nil {
s.internal(w, r, "create note", err)
return
}
writeJSON(w, http.StatusCreated, n)
}
func (s *Server) handleUpdateNote(w http.ResponseWriter, r *http.Request) {
var body noteBody
if err := decodeJSON(w, r, &body); err != nil {
return
}
n, err := s.db.Note(r.Context(), chi.URLParam(r, "noteID"))
if err != nil {
s.internal(w, r, "update note", err)
return
}
if n == nil || !s.mayTouchNote(r, n, store.RoleEditor) {
writeError(w, http.StatusNotFound, CodeNotFound, "no such note")
return
}
if body.ProjectID != nil {
// Moving it means being allowed on both sides: you cannot file somebody
// else's note into your project, nor yours into theirs.
if *body.ProjectID != "" {
if _, err := store.RequireProjectRole(r.Context(), s.db, *body.ProjectID,
userFrom(r.Context()).ID, store.RoleEditor); err != nil {
s.storeError(w, r, "move note", err)
return
}
}
n.ProjectID = *body.ProjectID
}
if body.Body != nil {
n.Body = *body.Body
}
if body.Pinned != nil {
n.Pinned = *body.Pinned
}
if err := s.db.SaveNote(r.Context(), n); err != nil {
s.internal(w, r, "update note", err)
return
}
writeJSON(w, http.StatusOK, n)
}
func (s *Server) handleDeleteNote(w http.ResponseWriter, r *http.Request) {
n, err := s.db.Note(r.Context(), chi.URLParam(r, "noteID"))
if err != nil {
s.internal(w, r, "delete note", err)
return
}
if n == nil || !s.mayTouchNote(r, n, store.RoleEditor) {
writeError(w, http.StatusNotFound, CodeNotFound, "no such note")
return
}
if err := s.db.DeleteNote(r.Context(), n.ID); err != nil {
s.internal(w, r, "delete note", err)
return
}
w.WriteHeader(http.StatusNoContent)
}
// handleNotesLinking answers the backwards question for a task or a project:
// what has somebody written about this.
func (s *Server) handleNotesLinking(w http.ResponseWriter, r *http.Request) {
kind := chi.URLParam(r, "kind")
if kind != "task" && kind != "project" && kind != "note" {
writeError(w, http.StatusBadRequest, CodeValidation, "kind must be task, project or note")
return
}
found, err := s.db.NotesLinking(r.Context(), kind, chi.URLParam(r, "targetID"))
if err != nil {
s.internal(w, r, "notes linking", err)
return
}
writeJSON(w, http.StatusOK, map[string]any{"notes": s.readable(r, found)})
}
// handleExportNotes writes every note this person can see as a zip of Markdown
// files, one per note.
//
// A zip of plain files rather than one bundle in a format of our own: what comes
// out is what was stored, and it opens in Obsidian, in a text editor, in anything.
// That is the promise the whole design was arranged around — the note on disk is
// already the file you would export — and this is where it is either kept or not.
func (s *Server) handleExportNotes(w http.ResponseWriter, r *http.Request) {
user := userFrom(r.Context())
notes, err := s.db.AllNotes(r.Context(), user.ID)
if err != nil {
s.internal(w, r, "export notes", err)
return
}
w.Header().Set("Content-Type", "application/zip")
w.Header().Set("Content-Disposition",
`attachment; filename="verdande-noter-`+time.Now().Format("2006-01-02")+`.zip"`)
zw := zip.NewWriter(w)
defer zw.Close()
// Names collide: two notes called "Møde" are two notes, and a zip with one entry
// twice is a zip that loses one of them.
used := map[string]int{}
for _, n := range notes {
name := noteFilename(n.Title)
used[name]++
if used[name] > 1 {
name = fmt.Sprintf("%s (%d)", name, used[name])
}
f, err := zw.CreateHeader(&zip.FileHeader{
Name: name + ".md",
Method: zip.Deflate,
Modified: n.UpdatedAt,
})
if err != nil {
// The response is already streaming, so there is no status left to send.
// Stopping leaves a truncated zip, which a zip tool says is truncated —
// better than a complete-looking file with a note missing from it.
s.log.Error("export notes", "err", err, "note", n.ID)
return
}
if _, err := io.WriteString(f, n.Body); err != nil {
s.log.Error("export notes", "err", err, "note", n.ID)
return
}
}
}
// noteFilename makes a title safe to be a filename on any of the three systems
// people actually use, without turning it into something unrecognisable.
func noteFilename(title string) string {
name := strings.TrimSpace(title)
if name == "" {
name = "uden titel"
}
// Windows refuses these outright; a slash would make directories on the others.
name = strings.Map(func(r rune) rune {
if strings.ContainsRune(`/\:*?"<>|`, r) || r < 0x20 {
return '-'
}
return r
}, name)
// Long titles are the first line of a note, which can be a whole sentence.
if len([]rune(name)) > 80 {
name = string([]rune(name)[:80])
}
return strings.TrimRight(name, " .")
}
// maxNoteImportBytes bounds one upload. Generous for text — a decade of notes is
// a few megabytes — and small enough that a wrong file is refused rather than read.
const maxNoteImportBytes = 64 << 20
// What the archive may become once it is unpacked, and how many entries it may
// hold. See handleImportNotes: the upload cap bounds what arrives, and says
// nothing at all about what it expands to.
const (
maxNoteImportUnpacked = 2 << 30
maxNoteImportFiles = 20_000
)
// handleImportNotes reads a zip of Markdown files and makes a note of each.
//
// The mirror of the export, and deliberately the same shape: whatever comes out of
// this program goes back into it. It is also the way in from anywhere else, since
// a folder of Markdown is what Obsidian, Bear, iA Writer and a shell script all
// produce — and what the Apple Notes exporter in tools/ writes.
//
// Filenames are ignored. The title is the first line of the body, everywhere else
// in this program, and making it the filename here would be a second rule for the
// same thing that disagrees the moment somebody renames a file.
func (s *Server) handleImportNotes(w http.ResponseWriter, r *http.Request) {
r.Body = http.MaxBytesReader(w, r.Body, maxNoteImportBytes)
if err := r.ParseMultipartForm(4 << 20); err != nil {
writeError(w, http.StatusRequestEntityTooLarge, CodePayloadTooLarge, "the file is too large")
return
}
defer r.MultipartForm.RemoveAll()
file, header, err := r.FormFile("file")
if err != nil {
writeFieldErrors(w, map[string]string{"file": "required"})
return
}
defer file.Close()
zr, err := zip.NewReader(file.(io.ReaderAt), header.Size)
if err != nil {
writeFieldErrors(w, map[string]string{"file": "not a zip archive"})
return
}
user := userFrom(r.Context())
created, skipped, files := 0, 0, 0
// Two passes. Everything that is not Markdown is stored first, so that when a
// note is read its pictures already have addresses to be rewritten to — a note
// and its images arrive in whatever order the archive happens to hold them, and
// one pass would leave half the links pointing at nothing.
// The archive as a whole, not just the upload.
//
// The 64 MB cap is on what arrives; deflate reaches about 1032:1, so those 64
// MB can become sixty-six gigabytes on the way out — written into the same
// volume the database lives on. The upload limit says nothing about that, and
// nothing else did either.
//
// Two ceilings: one per file, and one across the archive. A single enormous
// photograph and ten thousand small ones are different attacks with the same
// ending.
if len(zr.File) > maxNoteImportFiles {
writeError(w, http.StatusRequestEntityTooLarge, CodePayloadTooLarge,
"that archive holds too many files")
return
}
budget := int64(maxNoteImportUnpacked)
stored := map[string]string{}
for _, f := range zr.File {
name := f.Name
if f.FileInfo().IsDir() || skipFile(name) || strings.EqualFold(path.Ext(name), ".md") {
continue
}
// The header's own claim first, which costs nothing to check and rejects
// the obvious case before a byte is written. It is only a claim, so the
// reader below is limited as well.
if f.UncompressedSize64 > maxUploadBytes {
s.log.Warn("import: a file in the archive is too large", "file", name)
skipped++
continue
}
rc, err := f.Open()
if err != nil {
continue
}
hashed, size, err := s.storeUpload(io.LimitReader(rc, min(budget, maxUploadBytes)+1))
rc.Close()
if err != nil {
s.log.Warn("import note file", "err", err, "file", name)
continue
}
if size > maxUploadBytes || size > budget {
// Over the line once it was actually read: remove what was written
// rather than keeping a truncated file that would open as nothing.
os.Remove(filepath.Join(s.cfg.FilesDir(), hashed))
s.log.Warn("import: archive is larger unpacked than allowed", "file", name)
skipped++
continue
}
budget -= size
stored[name] = hashed
files++
}
for _, f := range zr.File {
name := f.Name
if f.FileInfo().IsDir() || !strings.EqualFold(path.Ext(name), ".md") {
continue
}
// Counted, not passed over in silence. A Markdown file the filter refuses is
// a note that did not arrive, and the caller is told the same number either
// way unless it is said here — which is how one note went missing without
// anything looking wrong.
if skipFile(name) {
s.log.Warn("import skipped a note file", "file", name)
skipped++
continue
}
rc, err := f.Open()
if err != nil {
skipped++
continue
}
body, err := io.ReadAll(io.LimitReader(rc, 2<<20))
rc.Close()
if err != nil || strings.TrimSpace(string(body)) == "" {
skipped++
continue
}
text, wrote, changed := readFrontMatter(string(body))
n := &store.Note{CreatedBy: user.ID, Body: text}
if err := s.db.SaveNote(r.Context(), n); err != nil {
s.log.Warn("import note", "err", err, "file", name)
skipped++
continue
}
// After the save, not as part of it: SaveNote owns `updated_at` and is right
// to, so the dates a note had before this program existed are written over
// the top rather than passed through it.
if !wrote.IsZero() || !changed.IsZero() {
if err := s.db.SetNoteTimes(r.Context(), n.ID, wrote, changed); err != nil {
s.log.Warn("import note dates", "err", err, "file", name)
}
}
// The pictures the note referred to, now that it has an id to hang them on.
// The text is rewritten to the addresses they were given, so an image that
// sat in the middle of a paragraph is still in the middle of it.
if rewritten, attached := s.attachToNote(r, n, path.Dir(name), stored, zr); attached > 0 {
n.Body = rewritten
if err := s.db.SaveNote(r.Context(), n); err != nil {
s.log.Warn("rewrite note links", "err", err, "note", n.ID)
}
}
created++
}
writeJSON(w, http.StatusOK, map[string]any{
"created": created, "skipped": skipped, "files": files,
})
}
// skipFile is the things in an archive that are not content: directories' own
// entries, macOS resource forks, and anything trying to climb out of the archive.
// A zip made on a Mac carries a __MACOSX copy of every file, and importing those
// would double a whole library with each copy full of binary rubbish.
func skipFile(name string) bool {
if strings.HasPrefix(name, "__MACOSX/") ||
strings.HasPrefix(path.Base(name), "._") ||
strings.HasPrefix(path.Base(name), ".") {
return true
}
// Traversal, read as path segments rather than as a substring.
//
// This was `strings.Contains(name, "..")`, which is the right idea aimed at the
// wrong thing: it does not describe a path that climbs out of the archive, it
// describes any name with two dots next to each other anywhere in it. A note
// called "Så blev det endelig jul! Stay tuned... ☕️" was therefore dropped —
// silently, and without even counting as skipped, so the number the import
// reported still looked right. One note in twelve hundred, found only by
// counting what came out.
//
// An ellipsis in a title is ordinary. A segment that *is* "..", or a path that
// starts at the root, is not — and those are what a zip uses to write outside
// the folder it was unpacked into.
if strings.HasPrefix(name, "/") {
return true
}
for _, part := range strings.Split(name, "/") {
if part == ".." {
return true
}
}
return false
}
// attachToNote records the files a note's text points at and rewrites the text to
// where they now live. Returns the new body and how many were attached.
//
// Only the files this note refers to. An archive is one folder for many notes, and
// hanging every picture on every note would turn a library of a hundred into ten
// thousand attachments.
func (s *Server) attachToNote(
r *http.Request, n *store.Note, dir string, stored map[string]string, zr *zip.Reader,
) (string, int) {
body := n.Body
attached := 0
for _, f := range zr.File {
hashed, ok := stored[f.Name]
if !ok {
continue
}
base := path.Base(f.Name)
// The three ways a Markdown file names a neighbouring image: by its path
// inside the archive, by the path relative to the note, and by its bare name.
//
// The archive path was missing, and it is the one an Apple Notes export
// actually writes — `vedhaeftninger/751-2-IMG_0615.heic`, with the note at the
// root beside the folder. Only the bare name matched, so only the bare name
// was replaced, and every picture in every imported note ended up pointing at
// `vedhaeftninger//api/v1/attachments/…`, which is nothing. The images were
// stored and attached correctly; it was the link that was wrong.
relative := strings.TrimPrefix(path.Join(dir, base), "./")
refs := []string{f.Name, relative, base}
if !strings.Contains(body, f.Name) &&
!strings.Contains(body, relative) &&
!strings.Contains(body, base) {
continue
}
a := &store.Attachment{
NoteID: n.ID,
Filename: base,
MimeType: mimeOf(base),
Size: int64(f.UncompressedSize64),
Path: hashed,
UploadedBy: n.CreatedBy,
}
if err := s.db.CreateAttachment(r.Context(), a); err != nil {
s.log.Warn("attach to note", "err", err, "file", base)
continue
}
// Længste først. Erstattes det bare navn inde i en længere sti, bliver mappen
// stående foran adressen, og linket peger på ingenting.
url := "/api/v1/attachments/" + a.ID
sort.Slice(refs, func(i, j int) bool { return len(refs[i]) > len(refs[j]) })
for _, ref := range refs {
if ref != "" {
body = strings.ReplaceAll(body, ref, url)
}
}
attached++
}
return body, attached
}
// readFrontMatter takes the YAML block off the top of a Markdown file and reads
// the two dates out of it, returning the rest of the text.
//
// Not a YAML parser, and deliberately not: the block this reads is the one this
// program writes, plus whatever Obsidian and Bear write, and all of them use the
// same handful of `key: value` lines. Pulling in a parser to read two dates would
// be a dependency that can fail in ways nobody here would recognise.
//
// A file with no front matter comes back untouched — that is the normal case, and
// it must never lose its first line to a rule about a block that is not there.
func readFrontMatter(body string) (text string, created, modified time.Time) {
if !strings.HasPrefix(body, "---\n") && !strings.HasPrefix(body, "---\r\n") {
return body, time.Time{}, time.Time{}
}
rest := body[strings.Index(body, "\n")+1:]
end := strings.Index(rest, "\n---")
if end < 0 {
// An opening fence with no close is not front matter; it is a horizontal
// rule somebody wrote at the top of their note.
return body, time.Time{}, time.Time{}
}
block := rest[:end]
after := rest[end+len("\n---"):]
after = strings.TrimPrefix(after, "\r")
after = strings.TrimPrefix(after, "\n")
for _, line := range strings.Split(block, "\n") {
key, value, ok := strings.Cut(line, ":")
if !ok {
continue
}
value = strings.TrimSpace(strings.Trim(strings.TrimSpace(value), `"'`))
switch strings.ToLower(strings.TrimSpace(key)) {
case "created", "created_at", "date":
created = parseNoteTime(value)
case "modified", "updated", "updated_at":
modified = parseNoteTime(value)
}
}
return strings.TrimLeft(after, "\n"), created, modified
}
// parseNoteTime accepts the shapes these files actually carry. A date without a
// time is that day at midnight, local — the note was written on that day, and
// pretending to know the hour would be worse than not having one.
func parseNoteTime(v string) time.Time {
for _, layout := range []string{
time.RFC3339,
"2006-01-02T15:04:05",
"2006-01-02 15:04:05",
"2006-01-02 15:04",
"2006-01-02",
} {
if t, err := time.ParseInLocation(layout, v, time.Local); err == nil {
return t
}
}
return time.Time{}
}
// mimeOf guesses from the extension. The archive does not carry a type, and
// sniffing the bytes would mean reading every file twice.
func mimeOf(name string) string {
if t := mime.TypeByExtension(path.Ext(name)); t != "" {
return t
}
return "application/octet-stream"
}
// handleUploadNoteAttachment takes a file into a note.
//
// The import could already do this, and nothing else could: a note brought in
// from Apple Notes arrived with its pictures, and a note written here could not
// be given one. Pasting a screenshot into a note is the most ordinary thing there
// is, and it did nothing at all.
//
// Editor rights on the note, checked the same way every other write to it is. The
// answer for "no such note" and "not yours" is the same, so an id cannot be
// probed by watching which one comes back.
func (s *Server) handleUploadNoteAttachment(w http.ResponseWriter, r *http.Request) {
n, err := s.db.Note(r.Context(), chi.URLParam(r, "noteID"))
if err != nil {
s.internal(w, r, "get note", err)
return
}
if n == nil || !s.mayTouchNote(r, n, store.RoleEditor) {
writeError(w, http.StatusNotFound, CodeNotFound, "no such note")
return
}
r.Body = http.MaxBytesReader(w, r.Body, maxUploadBytes+(1<<20))
if err := r.ParseMultipartForm(8 << 20); err != nil {
writeError(w, http.StatusRequestEntityTooLarge, CodePayloadTooLarge,
"that file is too large (25 MB maximum)")
return
}
defer r.MultipartForm.RemoveAll()
file, header, err := r.FormFile("file")
if err != nil {
writeFieldErrors(w, map[string]string{"file": "required"})
return
}
defer file.Close()
if header.Size > maxUploadBytes {
writeError(w, http.StatusRequestEntityTooLarge, CodePayloadTooLarge,
"that file is too large (25 MB maximum)")
return
}
stored, size, err := s.storeUpload(file)
if err != nil {
s.internal(w, r, "store upload", err)
return
}
user := userFrom(r.Context())
a := &store.Attachment{
NoteID: n.ID,
Filename: safeFilename(header.Filename),
MimeType: detectMime(header.Filename, header.Header.Get("Content-Type")),
Size: size,
Path: stored,
UploadedBy: user.ID,
}
if err := s.db.CreateAttachment(r.Context(), a); err != nil {
os.Remove(filepath.Join(s.cfg.FilesDir(), stored))
s.internal(w, r, "record attachment", err)
return
}
writeJSON(w, http.StatusCreated, toAttachmentJSON(*a))
}
// --- putting notes away, one or many ---------------------------------------------
type noteBulkRequest struct {
IDs []string `json:"ids"`
Archived *bool `json:"archived"`
Delete bool `json:"delete"`
}
// maxNoteBulk bounds one call. Enough to select a screenful several times over,
// and few enough that the loop below cannot become somebody's afternoon.
const maxNoteBulk = 500
// handleNoteBulk archives, unarchives or deletes a set of notes in one request.
//
// One call rather than one per note, because the thing being asked is one thing:
// "put these away". Fifty separate requests would be fifty round trips, fifty
// chances for half of them to fail, and a list that redraws fifty times — and
// after an import of twelve hundred notes, selecting fifty is the ordinary case,
// not the extreme one.
//
// Each note is still checked on its own. A set is a convenience for the caller
// and must never become a way to touch something they could not have touched one
// at a time.
func (s *Server) handleNoteBulk(w http.ResponseWriter, r *http.Request) {
var req noteBulkRequest
if err := decodeJSON(w, r, &req); err != nil {
return
}
if len(req.IDs) == 0 {
writeFieldErrors(w, map[string]string{"ids": "required"})
return
}
if len(req.IDs) > maxNoteBulk {
writeFieldErrors(w, map[string]string{"ids": "too many at once"})
return
}
done, skipped := 0, 0
for _, id := range req.IDs {
n, err := s.db.Note(r.Context(), id)
if err != nil {
s.internal(w, r, "get note", err)
return
}
// Not an error, and not a 404 for the whole call: a set can contain a note
// somebody else deleted a second ago, and the other forty-nine should still
// be put away. Counted so the answer is honest about it.
if n == nil || !s.mayTouchNote(r, n, store.RoleEditor) {
skipped++
continue
}
switch {
case req.Delete:
err = s.db.DeleteNote(r.Context(), id)
case req.Archived != nil:
err = s.db.SetNoteArchived(r.Context(), id, *req.Archived)
default:
writeFieldErrors(w, map[string]string{"archived": "say archived or delete"})
return
}
if err != nil {
s.internal(w, r, "note bulk", err)
return
}
done++
}
writeJSON(w, http.StatusOK, map[string]any{"done": done, "skipped": skipped})
}
@@ -0,0 +1,38 @@
package httpapi
import "testing"
// En titel med tre prikker i er ikke et forsøg på at skrive uden for mappen.
//
// `strings.Contains(name, "..")` var den rigtige idé rettet mod det forkerte:
// den beskriver ikke en sti, der klatrer ud af arkivet, men ethvert navn med to
// prikker ved siden af hinanden. Noten "Så blev det endelig jul! Stay tuned... ☕️"
// blev derfor droppet lydløst — én ud af tolv hundrede, og kun fundet ved at
// tælle efter, hvor mange der kom ud.
func TestSkipFileTakesEllipsesButRefusesTraversal(t *testing.T) {
keep := []string{
"Så blev det endelig jul! Stay tuned... ☕️.md",
"Møde 12.8..md",
"vedhaeftninger/1-2-Pasted Graphic.png",
"noter/Referat....md",
}
for _, name := range keep {
if skipFile(name) {
t.Errorf("skipFile(%q) = true, men det er et almindeligt filnavn", name)
}
}
drop := []string{
"../uden for.md",
"noter/../../etc/passwd",
"/etc/passwd",
"__MACOSX/._noget",
".skjult.md",
"noter/._AppleDouble.md",
}
for _, name := range drop {
if !skipFile(name) {
t.Errorf("skipFile(%q) = false, men den skal afvises", name)
}
}
}
@@ -0,0 +1,77 @@
package httpapi
import (
"archive/zip"
"bytes"
"io"
"testing"
)
// En zip-bombe skal ikke kunne skrive gigabyte ned på den disk, databasen ligger
// på.
//
// Uploaden er begrænset til 64 MB, og det siger ingenting om, hvad de 64 MB
// bliver til: deflate når omkring 1032:1, så de kan pakkes ud til seksogtres
// gigabyte. Prøven bygger en rigtig lille bombe — fire megabyte nuller, som
// pakker til nogle få kilobyte — og ser efter, at loftet på det udpakkede holder.
//
// Skrevet imod tallene frem for imod endepunktet, fordi det, der skal holde, er
// forholdet: en fil, hvis udpakkede størrelse overstiger loftet, må ikke læses
// færdig.
func TestAZipBombIsRefusedByItsUnpackedSize(t *testing.T) {
var buf bytes.Buffer
zw := zip.NewWriter(&buf)
w, err := zw.Create("bombe.bin")
if err != nil {
t.Fatal(err)
}
// Nuller pakker næsten helt væk. Fire megabyte er nok til at vise forholdet
// uden at prøven selv bruger en gigabyte.
if _, err := w.Write(make([]byte, 4<<20)); err != nil {
t.Fatal(err)
}
if err := zw.Close(); err != nil {
t.Fatal(err)
}
packed := int64(buf.Len())
if packed > 64<<10 {
t.Fatalf("prøvens egen bombe pakkede kun til %d bytes; den viser ikke noget", packed)
}
zr, err := zip.NewReader(bytes.NewReader(buf.Bytes()), packed)
if err != nil {
t.Fatal(err)
}
f := zr.File[0]
if f.UncompressedSize64 != 4<<20 {
t.Fatalf("udpakket størrelse = %d", f.UncompressedSize64)
}
// Det, importen gør: læs højst det, budgettet tillader, plus én byte — så en
// fil, der er præcis på grænsen, kan skelnes fra en, der er over.
rc, err := f.Open()
if err != nil {
t.Fatal(err)
}
defer rc.Close()
budget := int64(1 << 20)
n, err := io.Copy(io.Discard, io.LimitReader(rc, budget+1))
if err != nil {
t.Fatal(err)
}
if n <= budget {
t.Fatalf("læste %d bytes af en fil på 4 MB med et budget på %d — loftet fanger den ikke", n, budget)
}
}
// Tallene skal stå i et forhold, der giver mening: en enkelt fil må aldrig kunne
// bruge hele arkivets budget, og arkivet må ikke være mindre end én fil.
func TestTheImportCeilingsAgreeWithEachOther(t *testing.T) {
if maxNoteImportUnpacked <= maxUploadBytes {
t.Error("arkivets budget er ikke større end én enkelt fil")
}
if maxNoteImportFiles < 2000 {
t.Error("filloftet er under det, en rigtig Apple Noter-eksport indeholder")
}
}
+373
View File
@@ -0,0 +1,373 @@
package httpapi
import (
"archive/zip"
"bytes"
"encoding/json"
"io"
"mime/multipart"
"net/http"
"strings"
"testing"
)
// A note in a project is the project's; a note in no project is its author's.
// Both answers come from the project's own roles rather than from a rule notes
// invented for themselves, which is the point of filing them there.
func TestANoteIsNotReadableByJustAnybody(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
resp, body := ts.do(t, "POST", "/api/v1/notes", map[string]any{
"body": "Noget privat om #Firma",
})
if resp.StatusCode != http.StatusCreated {
t.Fatalf("create: %d %v", resp.StatusCode, body)
}
id, _ := body["id"].(string)
if id == "" {
t.Fatal("no id came back")
}
// The tags were read out of the text on the way in, without being asked for.
links, _ := body["links"].([]any)
if len(links) != 1 {
t.Fatalf("the note recorded %v", body["links"])
}
// Somebody else gets the same answer as for a note that does not exist, so an
// id cannot be probed for existence.
other := ts.newUser(t, "[email protected]", "Anden")
resp, _ = other.do(t, "GET", "/api/v1/notes/"+id, nil)
if resp.StatusCode != http.StatusNotFound {
t.Errorf("a stranger got %d for somebody else's note, want 404", resp.StatusCode)
}
}
// Searching must not become a way to read what you cannot open. The filter is
// applied after the index, and this is what says so.
func TestSearchDoesNotLeakNotesYouCannotRead(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
if resp, body := ts.do(t, "POST", "/api/v1/notes", map[string]any{
"body": "Hemmeligt kodeord til hjemmesiden",
}); resp.StatusCode != http.StatusCreated {
t.Fatalf("create: %d %v", resp.StatusCode, body)
}
_, mine := ts.do(t, "GET", "/api/v1/notes?q=hemmeligt", nil)
if got := len(mine["notes"].([]any)); got != 1 {
t.Fatalf("the author found %d of their own notes", got)
}
other := ts.newUser(t, "[email protected]", "Anden")
_, theirs := other.do(t, "GET", "/api/v1/notes?q=hemmeligt", nil)
if got := len(theirs["notes"].([]any)); got != 0 {
t.Errorf("a stranger's search returned %d notes", got)
}
}
// The backwards question, which is what makes notes worth linking at all.
func TestANoteIsFoundFromTheTaskItMentions(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
_, task := ts.do(t, "POST", "/api/v1/tasks/quick-add", map[string]any{
"text": "ring til Anders",
})
taskID, _ := task["id"].(string)
if taskID == "" {
t.Fatalf("no task id: %v", task)
}
if resp, _ := ts.do(t, "POST", "/api/v1/notes", map[string]any{
"body": "Aftalt at han ringer først, se /opgave/" + taskID,
}); resp.StatusCode != http.StatusCreated {
t.Fatal("the note was not written")
}
_, found := ts.do(t, "GET", "/api/v1/notes/linking/task/"+taskID, nil)
notes, _ := found["notes"].([]any)
if len(notes) != 1 {
t.Fatalf("%d notes point at the task, want 1", len(notes))
}
}
// Editing the text moves the links with it. A link that outlives its mention
// claims a connection that is not there.
func TestChangingANoteChangesWhatItPointsAt(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
_, note := ts.do(t, "POST", "/api/v1/notes", map[string]any{"body": "Om #Firma"})
id, _ := note["id"].(string)
_, before := ts.do(t, "GET", "/api/v1/notes/linking/project/Firma", nil)
if len(before["notes"].([]any)) != 1 {
t.Fatal("the first save recorded nothing")
}
if resp, _ := ts.do(t, "PATCH", "/api/v1/notes/"+id, map[string]any{
"body": "Om #Regnskab i stedet",
}); resp.StatusCode != http.StatusOK {
t.Fatal("the note was not changed")
}
_, after := ts.do(t, "GET", "/api/v1/notes/linking/project/Firma", nil)
if got := len(after["notes"].([]any)); got != 0 {
t.Errorf("%d notes still point at Firma", got)
}
}
// The export is the promise the whole design was arranged around: the note on
// disk is already the file you would export, so nothing is converted on the way
// out and nothing can be lost in the conversion.
func TestNotesExportAsMarkdownFiles(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
bodies := []string{
"# Møde med Anders\n\nHan vil gerne have **kaffe** hver uge.",
"Slash/i/titlen\n\nog noget tekst",
"# Møde med Anders\n\nen anden note med samme titel",
}
for _, body := range bodies {
if resp, _ := ts.do(t, "POST", "/api/v1/notes", map[string]any{"body": body}); resp.StatusCode != http.StatusCreated {
t.Fatalf("create: %d", resp.StatusCode)
}
}
req, _ := http.NewRequest("GET", ts.URL+"/api/v1/export/notes.zip", nil)
req.Header.Set("Sec-Fetch-Site", "same-origin")
for _, c := range ts.client.Jar.Cookies(mustParse(t, ts.URL)) {
req.AddCookie(c)
}
resp, err := ts.client.Do(req)
if err != nil {
t.Fatal(err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
t.Fatalf("export: status %d", resp.StatusCode)
}
if ct := resp.Header.Get("Content-Type"); ct != "application/zip" {
t.Errorf("content type is %q", ct)
}
raw, err := io.ReadAll(resp.Body)
if err != nil {
t.Fatal(err)
}
zr, err := zip.NewReader(bytes.NewReader(raw), int64(len(raw)))
if err != nil {
t.Fatalf("the archive does not open: %v", err)
}
if len(zr.File) != 3 {
t.Fatalf("the archive holds %d files, want 3", len(zr.File))
}
byName := map[string]string{}
for _, f := range zr.File {
r, err := f.Open()
if err != nil {
t.Fatal(err)
}
text, _ := io.ReadAll(r)
r.Close()
byName[f.Name] = string(text)
}
// Exactly what was stored, byte for byte. A converted export is an export that
// can be wrong.
if got := byName["Møde med Anders.md"]; got != bodies[0] {
t.Errorf("the note came out as %q", got)
}
// A slash in a title would make a directory, and on Windows it is refused
// outright.
if _, ok := byName["Slash-i-titlen.md"]; !ok {
t.Errorf("the awkward title became %v", keysOf(byName))
}
// Two notes with the same title are two notes; one entry twice is one lost.
if _, ok := byName["Møde med Anders (2).md"]; !ok {
t.Errorf("the second note of the same name is missing: %v", keysOf(byName))
}
}
func keysOf(m map[string]string) []string {
out := make([]string, 0, len(m))
for k := range m {
out = append(out, k)
}
return out
}
// Out and back in again. If an export cannot be imported, it is not an export —
// it is a file somebody hopes will be readable later.
func TestNotesSurviveTheRoundTripThroughAZip(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
var archive bytes.Buffer
zw := zip.NewWriter(&archive)
files := map[string]string{
"Møde med Anders.md": "# Møde med Anders\n\nHan vil have **kaffe** hver uge. #Firma",
"noget andet.md": "Løse tanker\n\nog en linje til",
// A zip made on a Mac carries these beside every file. Importing them would
// double the whole library, with each copy full of binary rubbish.
"__MACOSX/._Møde med Anders.md": "\x00\x05\x16\x07binært",
"billede.png": "ikke en note",
"tom.md": " \n ",
}
for name, body := range files {
f, err := zw.Create(name)
if err != nil {
t.Fatal(err)
}
if _, err := io.WriteString(f, body); err != nil {
t.Fatal(err)
}
}
if err := zw.Close(); err != nil {
t.Fatal(err)
}
var form bytes.Buffer
mw := multipart.NewWriter(&form)
part, _ := mw.CreateFormFile("file", "noter.zip")
part.Write(archive.Bytes())
mw.Close()
req, _ := http.NewRequest("POST", ts.URL+"/api/v1/notes/import", &form)
req.Header.Set("Content-Type", mw.FormDataContentType())
req.Header.Set("Sec-Fetch-Site", "same-origin")
for _, c := range ts.client.Jar.Cookies(mustParse(t, ts.URL)) {
req.AddCookie(c)
}
resp, err := ts.client.Do(req)
if err != nil {
t.Fatal(err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
t.Fatalf("import: status %d", resp.StatusCode)
}
var out struct{ Created, Skipped int }
json.NewDecoder(resp.Body).Decode(&out)
if out.Created != 2 {
t.Fatalf("imported %d notes, want the 2 real ones", out.Created)
}
// The title came from the first line, not from the filename — which is the same
// rule as everywhere else, and the reason renaming a file cannot rename a note.
_, listed := ts.do(t, "GET", "/api/v1/notes", nil)
titles := map[string]bool{}
for _, raw := range listed["notes"].([]any) {
titles[raw.(map[string]any)["title"].(string)] = true
}
if !titles["Møde med Anders"] || !titles["Løse tanker"] {
t.Errorf("the titles came out as %v", titles)
}
// And the #tag was read on the way in, so an imported note lands on its project
// exactly as a typed one does.
_, linked := ts.do(t, "GET", "/api/v1/notes/linking/project/Firma", nil)
if len(linked["notes"].([]any)) != 1 {
t.Errorf("the imported note did not link to its project")
}
}
// A note from Apple Notes is full of photographs. One that arrives as words alone
// is not that note — it is a summary of it.
func TestAnImportedNoteKeepsItsPictures(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
var archive bytes.Buffer
zw := zip.NewWriter(&archive)
write := func(name, body string) {
f, err := zw.Create(name)
if err != nil {
t.Fatal(err)
}
if _, err := io.WriteString(f, body); err != nil {
t.Fatal(err)
}
}
write("noter/Kvittering.md", "# Kvittering\n\nHer er den:\n\n![](vedhaeftninger/bon.png)\n\nog resten af teksten")
write("noter/vedhaeftninger/bon.png", "\x89PNG\r\n\x1a\nlad som om")
// Another note in the same archive, which must not inherit the first one's
// picture: an archive is one folder for many notes.
write("noter/Uden billede.md", "Uden billede\n\nbare tekst")
if err := zw.Close(); err != nil {
t.Fatal(err)
}
var form bytes.Buffer
mw := multipart.NewWriter(&form)
part, _ := mw.CreateFormFile("file", "noter.zip")
part.Write(archive.Bytes())
mw.Close()
req, _ := http.NewRequest("POST", ts.URL+"/api/v1/notes/import", &form)
req.Header.Set("Content-Type", mw.FormDataContentType())
req.Header.Set("Sec-Fetch-Site", "same-origin")
for _, c := range ts.client.Jar.Cookies(mustParse(t, ts.URL)) {
req.AddCookie(c)
}
resp, err := ts.client.Do(req)
if err != nil {
t.Fatal(err)
}
defer resp.Body.Close()
var out struct{ Created, Skipped, Files int }
json.NewDecoder(resp.Body).Decode(&out)
if out.Created != 2 || out.Files != 1 {
t.Fatalf("imported %d notes and %d files, want 2 and 1", out.Created, out.Files)
}
// The list carries a preview, not the whole note — see briefly(). The body is
// fetched by id, which is what the interface does when it opens one.
_, listed := ts.do(t, "GET", "/api/v1/notes?q=kvittering", nil)
notes := listed["notes"].([]any)
if len(notes) != 1 {
t.Fatalf("found %d notes", len(notes))
}
noteID := notes[0].(map[string]any)["id"].(string)
if got := notes[0].(map[string]any)["body"]; got != nil && got != "" {
t.Errorf("the list carried a body: %v — an editor handed that would save the truncation", got)
}
_, whole := ts.do(t, "GET", "/api/v1/notes/"+noteID, nil)
body := whole["body"].(string)
// The link points at the file where it now lives, not at a path inside an
// archive nobody kept.
if strings.Contains(body, "vedhaeftninger/bon.png") {
t.Errorf("the link still points into the archive: %q", body)
}
if !strings.Contains(body, "/api/v1/attachments/") {
t.Errorf("the picture was not attached: %q", body)
}
// And it is still where it was written, in the middle of the note.
if !strings.HasPrefix(body, "# Kvittering") || !strings.Contains(body, "og resten af teksten") {
t.Errorf("the note was rearranged: %q", body)
}
// The picture is really there and really readable.
id := body[strings.Index(body, "/api/v1/attachments/")+len("/api/v1/attachments/"):]
id = strings.SplitN(id, ")", 2)[0]
got, _ := ts.do(t, "GET", "/api/v1/attachments/"+id, nil)
if got.StatusCode != http.StatusOK {
t.Errorf("the attachment answers %d", got.StatusCode)
}
// The other note did not inherit it.
_, other := ts.do(t, "GET", "/api/v1/notes?q=Uden", nil)
for _, raw := range other["notes"].([]any) {
if strings.Contains(raw.(map[string]any)["body"].(string), "attachments/") {
t.Error("a note that mentioned no picture was given one")
}
}
}
+29 -1
View File
@@ -2,11 +2,13 @@ package httpapi
import (
"net/http"
"strings"
"time"
"github.com/go-chi/chi/v5"
"github.com/kristianwind/verdande/internal/push"
"github.com/kristianwind/verdande/internal/safedial"
"github.com/kristianwind/verdande/internal/store"
)
@@ -98,6 +100,12 @@ func (s *Server) notifyProject(r *http.Request, projectID, taskID, actorID, kind
}
}
// PushToUser is pushToUser for callers outside this package — the reminder job,
// which has the schedule but not the keys.
func (s *Server) PushToUser(userID, title, body, projectID string) {
s.pushToUser(userID, title, body, projectID)
}
func (s *Server) pushToUser(userID, title, body, projectID string) {
ctx, cancel := contextWithBackgroundTimeout(30 * time.Second)
defer cancel()
@@ -164,6 +172,23 @@ func (s *Server) handleSubscribePush(w http.ResponseWriter, r *http.Request) {
return
}
// The endpoint is an address this server will POST to, on a schedule the
// caller chooses — set a reminder a minute out and the push job fetches it,
// again every minute. Unchecked, that is a way to make the instance reach
// anything it can route to, on demand.
//
// https only: a push service is on the public internet and speaks TLS. Nothing
// legitimate needs http here, and allowing it is the difference between a
// blind request and a readable one.
if !strings.HasPrefix(req.Endpoint, "https://") {
writeFieldErrors(w, map[string]string{"endpoint": "must be an https address"})
return
}
if reason := safedial.CheckURL(req.Endpoint); reason != "" {
writeFieldErrors(w, map[string]string{"endpoint": reason})
return
}
err := s.db.SavePushSubscription(r.Context(), &store.PushSubscription{
UserID: userFrom(r.Context()).ID,
Endpoint: req.Endpoint,
@@ -183,7 +208,10 @@ func (s *Server) handleUnsubscribePush(w http.ResponseWriter, r *http.Request) {
if err := decodeJSON(w, r, &req); err != nil {
return
}
if err := s.db.DeletePushSubscription(r.Context(), req.Endpoint); err != nil {
// Scoped to the caller. It was not, so anybody who learned somebody else's
// endpoint could unsubscribe their device — a small thing to do to somebody
// and no reason at all to be able to.
if err := s.db.DeletePushSubscription(r.Context(), userFrom(r.Context()).ID, req.Endpoint); err != nil {
s.internal(w, r, "delete push subscription", err)
return
}
+16 -2
View File
@@ -4,6 +4,7 @@ import (
"context"
"fmt"
"net/http"
"strings"
"time"
)
@@ -72,13 +73,26 @@ func (s *Server) handleRestartFromPanel(w http.ResponseWriter, r *http.Request)
ctx, cancel := context.WithTimeout(context.WithoutCancel(r.Context()), 10*time.Second)
defer cancel()
url := fmt.Sprintf("%s/api/servers/%s/restart", s.cfg.PanelURL, s.cfg.PanelServerID)
req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, nil)
// safe-restart rather than restart, and the difference is not politeness.
//
// The plain endpoint does its work while the request is open: it stops the
// container — which kills this process — and the panel, seeing the caller
// vanish, never gets to the starting half. That took production down and left
// it down for four minutes on 18 August, and it cannot be fixed from this side:
// any button that synchronously asks for its own death has the same shape.
//
// safe-restart is scheduled. The panel answers "Restart scheduled" and does the
// work on its own afterwards, so nothing depends on this process still being
// alive to hear the reply.
url := fmt.Sprintf("%s/api/servers/%s/safe-restart", s.cfg.PanelURL, s.cfg.PanelServerID)
body := strings.NewReader(`{"backup_first":false,"target_id":""}`)
req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, body)
if err != nil {
s.internal(w, r, "build panel request", err)
return
}
req.Header.Set("Authorization", "Bearer "+s.cfg.PanelToken)
req.Header.Set("Content-Type", "application/json")
resp, err := (&http.Client{Timeout: 10 * time.Second}).Do(req)
if err != nil {
+74
View File
@@ -1,9 +1,11 @@
package httpapi
import (
"io"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
"time"
@@ -15,6 +17,11 @@ import (
// request has to end either way, or whatever sits in front answers for it.
func TestAPanelThatNeverAnswersStillEndsTheRequest(t *testing.T) {
panel := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Drained first. A real server reads what it was sent, and one that does not
// never learns the caller has hung up — httptest.Server.Close then waits for
// a handler that will never return, and the whole binary times out instead
// of the one request.
_, _ = io.Copy(io.Discard, r.Body)
<-r.Context().Done()
}))
defer panel.Close()
@@ -56,6 +63,7 @@ func TestAPanelThatNeverAnswersStillEndsTheRequest(t *testing.T) {
// And a panel that says no must say so in this server's own words, not in a proxy's.
func TestAPanelThatRefusesSaysSoAsJSON(t *testing.T) {
panel := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, _ = io.Copy(io.Discard, r.Body)
w.WriteHeader(http.StatusForbidden)
}))
defer panel.Close()
@@ -75,3 +83,69 @@ func TestAPanelThatRefusesSaysSoAsJSON(t *testing.T) {
t.Errorf("the reply does not say what the panel answered: %v", body)
}
}
// Forty copies of one mail arrived in a single second because two runs read the
// same marker before either wrote a new one — somebody pressing "Fetch now" while
// the ten-minute sweep was already inside the same mailbox.
//
// The lock is what stops it, and a lock is only worth having if something tries
// the door: this runs the two concurrently on purpose.
func TestTwoRunsOfOneMailboxDoNotOverlap(t *testing.T) {
// A bare server: the lock needs nothing but its own zero value, and building a
// database and a session around it would only make the test slower to read.
srv := &Server{}
var inside, most int
var mu sync.Mutex
release := make(chan struct{})
// Stands in for a run: takes the lock, notes how many are inside, and holds it
// until told to let go.
run := func(done chan<- struct{}) {
unlock := srv.lockSync("box-1")
mu.Lock()
inside++
if inside > most {
most = inside
}
mu.Unlock()
<-release
mu.Lock()
inside--
mu.Unlock()
unlock()
done <- struct{}{}
}
first, second := make(chan struct{}), make(chan struct{})
go run(first)
go run(second)
// Long enough that both would be inside if nothing kept them apart.
time.Sleep(100 * time.Millisecond)
close(release)
<-first
<-second
if most != 1 {
t.Errorf("%d runs were inside the same mailbox at once; that is how one mail "+
"becomes forty tasks", most)
}
// And two different mailboxes must not wait for each other: one slow host is
// not everybody's problem.
a := srv.lockSync("box-a")
done := make(chan struct{})
go func() {
srv.lockSync("box-b")()
close(done)
}()
select {
case <-done:
case <-time.After(2 * time.Second):
t.Error("a second mailbox waited for the first")
}
a()
}
+2 -1
View File
@@ -23,6 +23,7 @@ type projectJSON struct {
SortOrder float64 `json:"sort_order"`
Role string `json:"role"`
MemberCount int `json:"member_count"`
OpenCount int `json:"open_count"`
Shared bool `json:"shared"`
// GroupID is the sidebar group the asking user has filed it under, empty for
// none. Always empty on a project shared with them: grouping is a column on
@@ -34,7 +35,7 @@ func toProjectJSON(p store.Project) projectJSON {
return projectJSON{
ID: p.ID, Name: p.Name, Color: p.Color, Icon: p.Icon, ViewMode: p.ViewMode,
OwnerID: p.OwnerID, IsInbox: p.IsInbox, Archived: p.Archived,
SortOrder: p.SortOrder, Role: string(p.Role), MemberCount: p.MemberCount,
SortOrder: p.SortOrder, Role: string(p.Role), MemberCount: p.MemberCount, OpenCount: p.OpenCount,
Shared: p.MemberCount > 1, GroupID: p.GroupID,
}
}
+108 -4
View File
@@ -11,6 +11,7 @@ import (
"os"
"path"
"strings"
"sync"
"time"
"github.com/go-chi/chi/v5"
@@ -40,6 +41,18 @@ type Server struct {
mcp *mcp.Server
updates *update.Checker
// syncing serialises reads of one mailbox against itself. Two runs at once —
// somebody pressing "Fetch now" while the ten-minute sweep is in the middle of
// the same mailbox — both start from the marker as it was, both fetch the same
// messages, and both make tasks out of them. That is how forty copies of one
// mail arrived in a single second.
//
// A map of mutexes rather than one lock: two people's mailboxes have nothing to
// say to each other, and a single lock would make one slow host everybody's
// problem. Never cleaned out, because the key is a mailbox id and there are as
// many of those as there are mailboxes.
syncing sync.Map
// Nil when the base URL is not something an authenticator will accept — a bare
// hostname with no scheme, say. The endpoints answer 503 rather than the
// process refusing to start: passkeys are one way in among three, and an
@@ -51,6 +64,11 @@ type Server struct {
// than the rest of the API.
loginLimiter *limiter
resetLimiter *limiter
// Second factors are counted against the *account*, not the address. An
// address bucket is defeated by anybody with more than one address, which is
// the ordinary case — and a six-digit code is only safe while guesses are
// scarce, so the scarcity has to be counted against the thing being attacked.
totpLimiter *limiter
// Computed once from the built index.html; see csp.go.
csp string
@@ -68,6 +86,10 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
hub: realtime.NewHub(log),
loginLimiter: newLimiter(10, 15*time.Minute),
resetLimiter: newLimiter(5, time.Hour),
// Tighter than the password bucket, and for a whole hour. Somebody typing
// their own code gets it right on the first or second try; ten in an hour is
// generous for a person and hopeless for a search through a million.
totpLimiter: newLimiter(10, time.Hour),
}
s.csp = contentSecurityPolicy(scriptHashes(web))
if w, err := auth.NewWebAuthn(cfg.BaseURL, "verdande"); err == nil {
@@ -120,6 +142,12 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
r.Group(func(r chi.Router) {
r.Use(s.requireAuth)
r.Get("/oauth/gmail/callback", s.handleGmailCallback)
// The calendar comes back on a URI of its own rather than sharing Gmail's.
// One callback deciding from stored state which of the two flows had
// returned would put the choice of *which token store to write* behind a
// lookup — and when that goes wrong it goes wrong by overwriting a working
// Gmail connection with calendar tokens.
r.Get("/oauth/calendar/callback", s.handleCalendarCallback)
})
r.Route("/caldav", s.caldavRoutes)
@@ -130,6 +158,19 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
})
// The beacon's receiving end, and the number it produces. Both without a
// session, because the caller is a stranger's installation — that is the
// whole feature. Rate limited like the login routes: it is the one endpoint
// anybody on the internet can write to, and a row per request is a row per
// request whether or not the request is honest.
//
// Refused with a 404 unless this instance has been told it is the collector,
// so an ordinary installation is not a spam sink that answers 200 to
// anything.
r.Method(http.MethodPost, "/beacon",
s.rateLimit(s.loginLimiter, http.HandlerFunc(s.handleBeaconPing)))
r.Get("/beacon/count", s.handleBeaconCount)
// Reachable without being signed in — these are how you become signed in.
r.Route("/auth", func(r chi.Router) {
r.Get("/setup", s.handleSetupState)
@@ -165,13 +206,10 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
r.Get("/me", s.handleMe)
r.Patch("/me", s.handleUpdateProfile)
r.Put("/sidebar-sections", s.handleSetSidebarSections)
r.Put("/nav-order", s.handleSetNavOrder)
r.Post("/logout", s.handleLogout)
r.Post("/password/change", s.handleChangePassword)
r.Post("/totp/setup", s.handleTOTPSetup)
r.Post("/totp/confirm", s.handleTOTPConfirm)
r.Post("/totp/disable", s.handleTOTPDisable)
r.Get("/recovery-codes", s.handleRecoveryCodesCount)
r.Post("/recovery-codes", s.handleRecoveryCodesRegenerate)
// Behind requireSession, like the API tokens and for the same
// reason: a leaked token must not be able to end the sessions of
@@ -181,6 +219,23 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
r.Get("/sessions", s.handleListSessions)
r.Delete("/sessions/{sessionID}", s.handleDeleteSession)
// Turning the second factor on, and the codes that get you back
// in. Sessions only, and setup asks for the password.
//
// These sat outside this group, which meant the door opened with
// a token and closed only with a password: `disable` has always
// asked for the password, `setup` asked for nothing. A stolen
// token could therefore fetch the secret, load it into its own
// authenticator, confirm it, and keep the only copy of the
// recovery codes. The owner's next login would stop at a code
// they do not have, and they could not turn it off without
// logging in first. A theft becomes a tenancy — the same
// sentence written next door about passkeys, and the same rule.
r.Post("/totp/setup", s.handleTOTPSetup)
r.Post("/totp/confirm", s.handleTOTPConfirm)
r.Post("/totp/disable", s.handleTOTPDisable)
r.Post("/recovery-codes", s.handleRecoveryCodesRegenerate)
// Registering a key, and the list of them. Sessions only, for
// the same reason: a leaked token must not be able to add its
// own way in — that would turn a theft into a tenancy.
@@ -260,6 +315,36 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
r.Get("/version", s.handleVersion)
// Mailboxes read over IMAP — iCloud, Fastmail, any ordinary host. Each
// belongs to the person who connected it, so there is no admin gate
// here and no shared registration: the credential is theirs.
// Notes. Listing and searching are the same endpoint because they answer
// the same question with different words; the backwards one — what has
// been written about this task — hangs off the thing being asked about.
r.Route("/notes", func(r chi.Router) {
r.Get("/", s.handleListNotes)
r.Post("/", s.handleCreateNote)
r.Post("/import", s.handleImportNotes)
r.Get("/linking/{kind}/{targetID}", s.handleNotesLinking)
r.Get("/{noteID}", s.handleGetNote)
r.Patch("/{noteID}", s.handleUpdateNote)
r.Delete("/{noteID}", s.handleDeleteNote)
// A picture pasted or dropped into a note. The import could bring
// files in and nothing else could, which made the most ordinary act
// in a notes app — paste a screenshot — do nothing at all.
r.Post("/{noteID}/attachments", s.handleUploadNoteAttachment)
// Mange på én gang. Efter en import af tolv hundrede noter er det at
// vælge halvtreds det almindelige tilfælde, ikke det yderste.
r.Post("/bulk", s.handleNoteBulk)
})
r.Route("/mailboxes", func(r chi.Router) {
r.Get("/", s.handleListMailboxes)
r.Post("/", s.handleAddMailbox)
r.Delete("/{mailboxID}", s.handleDeleteMailbox)
r.Post("/{mailboxID}/sync", s.handleSyncMailbox)
})
r.Route("/gmail", func(r chi.Router) {
r.Get("/", s.handleGetGmail)
r.Put("/", s.handleSetGmail)
@@ -279,12 +364,26 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
})
})
// A calendar read from somewhere else, laid under verdande's own dated
// tasks. Read-only, so there is no write here to gate: the events are a
// copy of somebody else's calendar, and the only thing a person decides
// is which of them to look at.
r.Route("/calendar", func(r chi.Router) {
r.Get("/", s.handleGetCalendar)
r.Put("/calendars", s.handleSetCalendars)
r.Delete("/", s.handleDisconnectCalendar)
r.Post("/authorize", s.handleCalendarAuthorize)
r.Post("/sync", s.handleCalendarSyncNow)
r.Get("/events", s.handleCalendarEvents)
})
r.Route("/import", func(r chi.Router) {
r.Post("/todoist", s.handleImportTodoist)
r.Post("/csv", s.handleImportCSV)
})
r.Get("/export/account", s.handleExportAccount)
r.Get("/export/notes.zip", s.handleExportNotes)
r.Get("/export/projects/{projectID}.csv", s.handleExportProject)
r.Get("/export/projects/{projectID}.ics", s.handleExportProjectICS)
@@ -352,6 +451,11 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
// `:latest` on the way, which is what makes a restart an update.
r.Get("/panel", s.handlePanelStatus)
r.Post("/panel/restart", s.handleRestartFromPanel)
// Who this instance reports to, and whether it does. An
// administrator's decision, not a person's: there is one
// installation and it either phones home or it does not.
r.Get("/beacon/settings", s.handleBeaconStatus)
r.Put("/beacon/settings", s.handleSetBeacon)
// A backup file is a complete copy of the database, so this whole
// group is sessions-only as well as administrators-only: a leaked
// token must not be able to download everybody's data.
+17
View File
@@ -20,7 +20,24 @@ type totpSetupResponse struct {
}
func (s *Server) handleTOTPSetup(w http.ResponseWriter, r *http.Request) {
// The password, the same as disable has always asked for.
//
// Symmetry is the point. Turning the second factor on is at least as
// consequential as turning it off — it decides which device gets to say yes
// from now on — and a door that opens more easily than it closes is not a
// door. Together with the move behind requireSession, this is what stops a
// stolen credential from enrolling its own authenticator and keeping the only
// copy of the recovery codes.
var req totpDisableRequest
if err := decodeJSON(w, r, &req); err != nil {
return
}
user := userFrom(r.Context())
if err := auth.VerifyPassword(user.PasswordHash, req.Password); err != nil {
writeFieldErrors(w, map[string]string{"password": "not correct"})
return
}
if user.TOTPEnabled {
writeError(w, http.StatusConflict, CodeConflict,
"two-factor authentication is already switched on")
+206 -23
View File
@@ -16,14 +16,18 @@
package imap
import (
"bytes"
"crypto/tls"
"crypto/x509"
"fmt"
"io"
"net"
"strings"
"time"
"github.com/emersion/go-imap/v2"
"github.com/emersion/go-imap/v2/imapclient"
"github.com/emersion/go-message"
)
// Message is one mail, reduced to what a task needs.
@@ -41,6 +45,13 @@ type Account struct {
Username string
Password string // an app-specific password on every host worth naming
Folder string // "INBOX" when empty
// RootCAs adds a certificate authority to trust, on top of the system's. It
// exists so a test can put a real IMAP server behind a self-signed certificate
// and still be refused by everything else — the field can only widen trust,
// never switch verification off, which is the difference between a seam and a
// hole. Nil in production, where the system's roots are the answer.
RootCAs *x509.CertPool
}
// Dial opens a TLS connection and logs in. The caller closes it.
@@ -55,7 +66,11 @@ func Dial(a Account) (*Client, error) {
}
c, err := imapclient.DialTLS(a.Host, &imapclient.Options{
TLSConfig: &tls.Config{ServerName: host, MinVersion: tls.VersionTLS12},
TLSConfig: &tls.Config{
ServerName: host,
MinVersion: tls.VersionTLS12,
RootCAs: a.RootCAs,
},
})
if err != nil {
return nil, fmt.Errorf("connect to %s: %w", a.Host, err)
@@ -96,9 +111,13 @@ func (c *Client) Close() error {
// that consumed it would be taking something away. Flagging a mail is a deliberate
// act with no other meaning, which is exactly what is wanted — the same reason the
// Gmail side reads stars.
func (c *Client) Since(uid uint32, limit int) ([]Message, error) {
// The second return is the highest uid the search considered. A caller that got
// through every message can move its marker to that, and then it does not matter
// whether the server put a uid in each fetch reply — which some do not, and which
// is how the same mail became a task again on every run.
func (c *Client) Since(uid uint32, limit int) ([]Message, uint32, error) {
if _, err := c.c.Select(c.folder, &imap.SelectOptions{ReadOnly: true}).Wait(); err != nil {
return nil, fmt.Errorf("open %s: %w", c.folder, err)
return nil, 0, fmt.Errorf("open %s: %w", c.folder, err)
}
// From uid+1 to the end. A mailbox with nothing new answers with no uids
@@ -112,12 +131,27 @@ func (c *Client) Since(uid uint32, limit int) ([]Message, error) {
}
found, err := c.c.UIDSearch(criteria, nil).Wait()
if err != nil {
return nil, fmt.Errorf("search %s: %w", c.folder, err)
return nil, 0, fmt.Errorf("search %s: %w", c.folder, err)
}
uids := found.AllUIDs()
// Anything at or below the marker is dropped here rather than trusted to the
// range in the query. A server that reads `1272:*` as `0:1272` — and one does,
// which is how the same mail became a task every ten minutes with the marker
// standing still at 1271 — hands back the whole mailbox. The marker is ours;
// enforcing it here costs one pass over a list of at most a few hundred
// numbers and does not depend on anybody else's reading of the spec.
kept := make([]imap.UID, 0, len(uids))
for _, u := range uids {
if uint32(u) > uid {
kept = append(kept, u)
}
}
uids = kept
if len(uids) == 0 {
return nil, nil
return nil, uid, nil
}
if limit > 0 && len(uids) > limit {
// The newest ones. A mailbox flagged over years would otherwise make the
@@ -127,29 +161,51 @@ func (c *Client) Since(uid uint32, limit int) ([]Message, error) {
want := imap.UIDSetNum(uids...)
fetch := &imap.FetchOptions{
Envelope: true,
UID: true,
BodySection: []*imap.FetchItemBodySection{{Specifier: imap.PartSpecifierText, Peek: true}},
Envelope: true,
UID: true,
// The whole message, headers included, so the MIME structure can be walked.
// TEXT alone gives the body as bytes — boundaries and part headers and all —
// which is what put "--=_be71cb39… Content-Transfer-Encoding: 8bit" at the
// front of every task made from a multipart mail.
BodySection: []*imap.FetchItemBodySection{{Peek: true}},
}
var highest uint32
for _, u := range uids {
if uint32(u) > highest {
highest = uint32(u)
}
}
msgs, err := c.c.Fetch(want, fetch).Collect()
if err != nil {
return nil, fmt.Errorf("fetch from %s: %w", c.folder, err)
return nil, 0, fmt.Errorf("fetch from %s: %w", c.folder, err)
}
out := make([]Message, 0, len(msgs))
for _, m := range msgs {
for i, m := range msgs {
if m.Envelope == nil {
continue
}
// The uid comes from the search when the fetch reply does not carry one.
// A missing uid is not cosmetic: it is the marker the next run starts from,
// and a marker stuck at zero makes every message arrive again on every run.
// That is what happened against a real server while the tests stayed green,
// because the server they run against always answers with it.
msgUID := uint32(m.UID)
if msgUID == 0 && i < len(uids) {
msgUID = uint32(uids[i])
}
out = append(out, Message{
UID: uint32(m.UID),
UID: msgUID,
From: address(m.Envelope.From),
Subject: strings.TrimSpace(m.Envelope.Subject),
Snippet: snippet(m),
Date: m.Envelope.Date,
})
}
return out, nil
return out, highest, nil
}
// address renders the first sender as "Name <addr>", or just the address.
@@ -165,23 +221,150 @@ func address(list []imap.Address) string {
return a.Name + " <" + addr + ">"
}
// snippet is the first readable line or so of the body.
// snippet is the first readable text in the mail.
//
// No MIME walk and no HTML stripping yet: this is what shows under a task's title,
// and a wrong-but-short line is a smaller problem than a parser that has to be
// right about every mail ever sent. The full mail stays where it is.
// It walks the MIME structure rather than taking the body as bytes. text/plain is
// preferred and HTML is the fallback with its tags stripped — crudely, because
// this is the line under a task's title and not a rendering engine. The mail
// itself stays where it is.
const snippetLimit = 400
func snippet(m *imapclient.FetchMessageBuffer) string {
for _, body := range m.BodySection {
text := strings.TrimSpace(string(body.Bytes))
if text == "" {
continue
if text := readable(body.Bytes); text != "" {
return cut(text)
}
if len(text) > snippetLimit {
text = text[:snippetLimit] + "…"
}
return text
}
return ""
}
func readable(raw []byte) string {
entity, err := message.Read(bytes.NewReader(raw))
if err != nil {
// Not something this parser recognises. The raw bytes read badly but read,
// and a mail is not worth losing over its own headers.
return strings.TrimSpace(string(raw))
}
return strings.TrimSpace(fromEntity(entity, 0))
}
// fromEntity descends into a multipart mail. The depth guard is for the mail that
// nests parts inside parts inside parts — rare, and cheaper to bound than to trust.
func fromEntity(entity *message.Entity, depth int) string {
if depth > 4 {
return ""
}
if parts := entity.MultipartReader(); parts != nil {
var html string
for {
part, err := parts.NextPart()
if err != nil {
break
}
if text := fromEntity(part, depth+1); text != "" {
kind, _, _ := part.Header.ContentType()
if kind != "text/html" {
return text
}
if html == "" {
html = text
}
}
}
return html
}
body, _ := io.ReadAll(entity.Body)
kind, _, _ := entity.Header.ContentType()
if kind == "text/html" {
return stripTags(string(body))
}
return strings.TrimSpace(string(body))
}
// stripTags removes markup without pretending to understand it. Enough for one
// line of preview, and honest about being no more than that.
func stripTags(html string) string {
var b strings.Builder
depth := 0
for _, r := range html {
switch {
case r == '<':
depth++
case r == '>':
if depth > 0 {
depth--
}
case depth == 0:
b.WriteRune(r)
}
}
return strings.Join(strings.Fields(b.String()), " ")
}
func cut(text string) string {
runes := []rune(text)
if len(runes) > snippetLimit {
return string(runes[:snippetLimit]) + "…"
}
return text
}
// Unflag takes the flag off the messages that have become tasks.
//
// This is what stops the same mail arriving twice, and it replaces a marker that
// could not be made to hold. The marker was a uid: remember the highest one seen,
// start above it next time. That is correct only if uids are what the server says
// they are, and it broke in every way such a scheme can — a fetch reply without a
// uid, a UIDVALIDITY that changed under a rebuilt mailbox, two mailbox rows
// pointing at the same folder and each keeping its own marker. Every one of those
// produced the same symptom: the same mail became a task again, and again.
//
// Unflagging moves the record out of our database and into the mailbox, where the
// question is asked. "Flagged" means "turn this into a task"; once it is a task,
// the flag has done its job and is not true any more. Nothing has to be
// remembered, so there is nothing to get out of step — a second sync, a second
// mailbox row, or a database restored from a backup all find the same answer,
// because the answer is in the mailbox.
//
// It is destructive in the small way that matters: it changes something in
// somebody's mail. That is the trade, and it is the right way round — a flag is a
// note to self that this needs doing, and it has been done.
//
// The folder is re-selected writable. Since() opens it read-only on purpose, and
// a search should not need write access just because a later step might.
func (c *Client) Unflag(uids []uint32) error {
if len(uids) == 0 {
return nil
}
if _, err := c.c.Select(c.folder, nil).Wait(); err != nil {
return fmt.Errorf("open %s for writing: %w", c.folder, err)
}
set := imap.UIDSetNum()
for _, u := range uids {
if u == 0 {
// No uid means nothing to address. The message keeps its flag and will
// be seen again — which is the failure this whole function exists to
// prevent, so it is worth saying rather than skipping quietly.
continue
}
set.AddNum(imap.UID(u))
}
if len(set) == 0 {
return nil
}
// Silent: the server has nothing to tell us that we would act on, and the
// untagged replies would only have to be collected and thrown away.
cmd := c.c.Store(set, &imap.StoreFlags{
Op: imap.StoreFlagsDel,
Silent: true,
Flags: []imap.Flag{imap.FlagFlagged},
}, nil)
if err := cmd.Close(); err != nil {
return fmt.Errorf("unflag in %s: %w", c.folder, err)
}
return nil
}
+396
View File
@@ -0,0 +1,396 @@
package imap
import (
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"crypto/tls"
"crypto/x509"
"crypto/x509/pkix"
"encoding/pem"
"math/big"
"net"
"strings"
"testing"
"time"
"github.com/emersion/go-imap/v2"
"github.com/emersion/go-imap/v2/imapserver"
"github.com/emersion/go-imap/v2/imapserver/imapmemserver"
)
// The round trip, against a real IMAP server rather than a stub of one. The
// earlier tests cover the pure parts — what a sender renders as, where a snippet
// is cut — and would all have passed with the fetch itself broken.
func TestFlaggedMailComesBackAsMessages(t *testing.T) {
addr, pool := serveMailbox(t, []mail{
{subject: "Faktura 4711", from: "Anders Bo <[email protected]>", body: "Vedhæftet.", flagged: true},
{subject: "Nyhedsbrev", from: "[email protected]", body: "Læs mere", flagged: false},
{subject: "Kontrakt", from: "[email protected]", body: "Til underskrift", flagged: true},
})
client, err := Dial(Account{
Host: addr, Username: "kw", Password: "hemmelig", RootCAs: pool,
})
if err != nil {
t.Fatal(err)
}
defer client.Close()
msgs, _, err := client.Since(0, 25)
if err != nil {
t.Fatal(err)
}
// Two of the three. The newsletter is unflagged, and flagging is the whole
// signal: unread is a state the owner is still using.
if len(msgs) != 2 {
t.Fatalf("got %d messages, want the 2 flagged ones", len(msgs))
}
if msgs[0].Subject != "Faktura 4711" {
t.Errorf("first subject is %q", msgs[0].Subject)
}
if !strings.Contains(msgs[0].From, "[email protected]") {
t.Errorf("first sender is %q", msgs[0].From)
}
if msgs[0].UID == 0 {
t.Error("no uid came back, so a second run would fetch everything again")
}
// And reading on from a marker returns only what is newer, which is the whole
// of the deduplication.
rest, _, err := client.Since(msgs[0].UID, 25)
if err != nil {
t.Fatal(err)
}
if len(rest) != 1 || rest[0].Subject != "Kontrakt" {
t.Fatalf("reading on from uid %d gave %d messages", msgs[0].UID, len(rest))
}
}
func TestAWrongPasswordIsRefusedClearly(t *testing.T) {
addr, pool := serveMailbox(t, nil)
_, err := Dial(Account{Host: addr, Username: "kw", Password: "forkert", RootCAs: pool})
if err == nil {
t.Fatal("a wrong password was accepted")
}
// Hosts say everything from AUTHENTICATIONFAILED to a sentence about the web,
// so the reply names the host and the user rather than repeating their words.
if !strings.Contains(err.Error(), "kw") {
t.Errorf("the error does not say whose login was refused: %v", err)
}
}
// A certificate this process signed itself must not be enough on its own.
func TestAnUntrustedCertificateIsRefused(t *testing.T) {
addr, _ := serveMailbox(t, nil)
if _, err := Dial(Account{Host: addr, Username: "kw", Password: "hemmelig"}); err == nil {
t.Fatal("a self-signed certificate was accepted without being trusted")
}
}
type mail struct {
subject, from, body string
flagged bool
// raw replaces the whole message when set, which is the only way to get a
// genuinely multipart one in.
raw string
}
// serveMailbox starts an in-process IMAP server over TLS and returns its address
// and the pool that trusts it.
func serveMailbox(t *testing.T, mails []mail) (string, *x509.CertPool) {
t.Helper()
cert, pool := selfSigned(t)
memServer := imapmemserver.New()
user := imapmemserver.NewUser("kw", "hemmelig")
if err := user.Create("INBOX", nil); err != nil {
t.Fatal(err)
}
memServer.AddUser(user)
for _, m := range mails {
flags := []imap.Flag{}
if m.flagged {
flags = append(flags, imap.FlagFlagged)
}
raw := m.raw
if raw == "" {
raw = "From: " + m.from + "\r\nSubject: " + m.subject +
"\r\nDate: Tue, 18 Aug 2026 10:00:00 +0200\r\n\r\n" + m.body + "\r\n"
}
if _, err := user.Append("INBOX", strings.NewReader(raw), &imap.AppendOptions{
Flags: flags,
}); err != nil {
t.Fatal(err)
}
}
srv := imapserver.New(&imapserver.Options{
NewSession: func(_ *imapserver.Conn) (imapserver.Session, *imapserver.GreetingData, error) {
return memServer.NewSession(), nil, nil
},
TLSConfig: &tls.Config{Certificates: []tls.Certificate{cert}},
InsecureAuth: false,
})
t.Cleanup(func() { srv.Close() })
ln, err := tls.Listen("tcp", "127.0.0.1:0", &tls.Config{Certificates: []tls.Certificate{cert}})
if err != nil {
t.Fatal(err)
}
go srv.Serve(ln)
// "localhost", not the loopback address: the certificate names a host, and the
// client checks that name — as it should.
_, port, _ := net.SplitHostPort(ln.Addr().String())
return net.JoinHostPort("localhost", port), pool
}
func selfSigned(t *testing.T) (tls.Certificate, *x509.CertPool) {
t.Helper()
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
t.Fatal(err)
}
template := x509.Certificate{
SerialNumber: big.NewInt(1),
Subject: pkix.Name{CommonName: "localhost"},
DNSNames: []string{"localhost"},
IPAddresses: []net.IP{net.ParseIP("127.0.0.1")},
NotBefore: time.Now().Add(-time.Hour),
NotAfter: time.Now().Add(time.Hour),
KeyUsage: x509.KeyUsageDigitalSignature | x509.KeyUsageCertSign,
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
IsCA: true,
BasicConstraintsValid: true,
}
der, err := x509.CreateCertificate(rand.Reader, &template, &template, &key.PublicKey, key)
if err != nil {
t.Fatal(err)
}
certPEM := pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der})
keyDER, err := x509.MarshalECPrivateKey(key)
if err != nil {
t.Fatal(err)
}
keyPEM := pem.EncodeToMemory(&pem.Block{Type: "EC PRIVATE KEY", Bytes: keyDER})
cert, err := tls.X509KeyPair(certPEM, keyPEM)
if err != nil {
t.Fatal(err)
}
pool := x509.NewCertPool()
pool.AppendCertsFromPEM(certPEM)
return cert, pool
}
// The bug this was written for: every press of "Fetch now" made the same mail
// into a task again. The marker the next run starts from is the uid, and if it
// comes back as zero the marker never moves.
func TestEveryMessageCarriesAUID(t *testing.T) {
addr, pool := serveMailbox(t, []mail{
{subject: "En", from: "[email protected]", body: "en", flagged: true},
{subject: "To", from: "[email protected]", body: "to", flagged: true},
})
client, err := Dial(Account{Host: addr, Username: "kw", Password: "hemmelig", RootCAs: pool})
if err != nil {
t.Fatal(err)
}
defer client.Close()
msgs, _, err := client.Since(0, 25)
if err != nil {
t.Fatal(err)
}
for _, m := range msgs {
if m.UID == 0 {
t.Fatalf("%q came back without a uid; the marker would never move and "+
"every run would make it into a task again", m.Subject)
}
}
if msgs[0].UID == msgs[1].UID {
t.Error("two messages share a uid")
}
}
// The other half of the same report: the snippet under a task's title read
// "--=_be71cb39… Content-Transfer-Encoding: 8bit Content-Type: text/plain".
func TestAMultipartMailReadsAsItsText(t *testing.T) {
const boundary = "=_be71cb3975bdfbc60a16327cb2f49c23"
body := "This is a multi-part message.\r\n" +
"--" + boundary + "\r\n" +
"Content-Transfer-Encoding: 8bit\r\n" +
"Content-Type: text/plain; charset=UTF-8; format=flowed\r\n\r\n" +
"Hej igen,\r\n\r\nVi har normalt 3 slags. :)\r\n" +
"--" + boundary + "\r\n" +
"Content-Type: text/html; charset=UTF-8\r\n\r\n" +
"<html><body><p>Hej igen</p></body></html>\r\n" +
"--" + boundary + "--\r\n"
addr, pool := serveMailbox(t, []mail{{
flagged: true,
raw: "Content-Type: multipart/alternative; boundary=\"" + boundary + "\"\r\n" +
"From: [email protected]\r\nSubject: Re: Kaffebestilling\r\n" +
"Date: Tue, 18 Aug 2026 10:00:00 +0200\r\n\r\n" + body,
}})
client, err := Dial(Account{Host: addr, Username: "kw", Password: "hemmelig", RootCAs: pool})
if err != nil {
t.Fatal(err)
}
defer client.Close()
msgs, _, err := client.Since(0, 25)
if err != nil {
t.Fatal(err)
}
if len(msgs) != 1 {
t.Fatalf("got %d messages", len(msgs))
}
got := msgs[0].Snippet
for _, junk := range []string{boundary, "Content-Transfer-Encoding", "Content-Type", "<html>", "<p>"} {
if strings.Contains(got, junk) {
t.Errorf("the snippet still carries %q: %q", junk, got)
}
}
if !strings.HasPrefix(got, "Hej igen") {
t.Errorf("the snippet does not start with the text of the mail: %q", got)
}
}
// The marker is ours, not the server's.
//
// A mailbox read with a marker above everything in it must come back empty. The
// query says so, but a server is free to read the range differently — one reads
// `1272:*` as `0:1272` and hands back the lot, which is how the same mail became
// a task every ten minutes while the marker stood still at 1271. The filter is
// on this side now, so the invariant holds whatever comes back.
func TestNothingAtOrBelowTheMarkerComesBack(t *testing.T) {
addr, pool := serveMailbox(t, []mail{
{subject: "En", from: "[email protected]", body: "en", flagged: true},
{subject: "To", from: "[email protected]", body: "to", flagged: true},
{subject: "Tre", from: "[email protected]", body: "tre", flagged: true},
})
client, err := Dial(Account{Host: addr, Username: "kw", Password: "hemmelig", RootCAs: pool})
if err != nil {
t.Fatal(err)
}
defer client.Close()
all, highest, err := client.Since(0, 25)
if err != nil {
t.Fatal(err)
}
if len(all) != 3 {
t.Fatalf("got %d messages on the first read, want 3", len(all))
}
// Read again from the marker the first run would have written.
rest, again, err := client.Since(highest, 25)
if err != nil {
t.Fatal(err)
}
if len(rest) != 0 {
t.Errorf("%d message(s) came back at or below the marker: %v", len(rest), rest[0].Subject)
}
// And the marker must not go backwards, or the next run reads them all again.
if again < highest {
t.Errorf("the marker moved back from %d to %d", highest, again)
}
// Every message, individually: none of them may reappear once passed.
for _, m := range all {
later, _, err := client.Since(m.UID, 25)
if err != nil {
t.Fatal(err)
}
for _, l := range later {
if l.UID <= m.UID {
t.Errorf("reading from %d gave %d back", m.UID, l.UID)
}
}
}
}
// Den samme mail må ikke kunne blive en opgave to gange.
//
// Prøven kører hele vejen imod en rigtig server: den finder de flagede mails,
// tager flaget af dem, og spørger så forfra som en ny synkronisering ville — med
// markøren sat til nul, altså det værst tænkelige tilfælde, hvor alt, hvad vi
// selv havde husket, er væk. Der skal ikke komme noget tilbage.
//
// Det er det, markøren ikke kunne love. Den holdt kun, hvis uids var, hvad
// serveren sagde, og fejlede på et fetch-svar uden uid, på en UIDVALIDITY, der
// skiftede, og på to postkasse-rækker, der pegede samme sted og hver især førte
// deres eget regnskab. Alle tre gav det samme: mailen blev en opgave igen.
func TestAnUnflaggedMailDoesNotComeBack(t *testing.T) {
addr, pool := serveMailbox(t, []mail{
{subject: "Faktura 4711", from: "[email protected]", body: "Vedhæftet.", flagged: true},
{subject: "Nyhedsbrev", from: "[email protected]", body: "Læs mere", flagged: false},
{subject: "Kontrakt", from: "[email protected]", body: "Til underskrift", flagged: true},
})
client, err := Dial(Account{Host: addr, Username: "kw", Password: "hemmelig", RootCAs: pool})
if err != nil {
t.Fatal(err)
}
defer client.Close()
msgs, _, err := client.Since(0, 25)
if err != nil {
t.Fatal(err)
}
if len(msgs) != 2 {
t.Fatalf("fandt %d flagede mails, ventede 2", len(msgs))
}
uids := make([]uint32, 0, len(msgs))
for _, m := range msgs {
uids = append(uids, m.UID)
}
if err := client.Unflag(uids); err != nil {
t.Fatalf("Unflag: %v", err)
}
// Forfra, og fra nul: selv en instans, der intet husker, må ikke finde dem igen.
again, _, err := client.Since(0, 25)
if err != nil {
t.Fatal(err)
}
if len(again) != 0 {
t.Fatalf("%d mails kom igen efter at flaget var taget af — den første af dem er %q",
len(again), again[0].Subject)
}
}
// Et tomt kald må ikke åbne mappen skrivbar for ingenting.
func TestUnflagWithNothingToDoIsQuiet(t *testing.T) {
addr, pool := serveMailbox(t, []mail{
{subject: "Faktura", from: "[email protected]", body: "x", flagged: true},
})
client, err := Dial(Account{Host: addr, Username: "kw", Password: "hemmelig", RootCAs: pool})
if err != nil {
t.Fatal(err)
}
defer client.Close()
if err := client.Unflag(nil); err != nil {
t.Errorf("Unflag(nil) = %v", err)
}
// Kun nuller: der er intet at adressere, og mailen skal beholde sit flag.
if err := client.Unflag([]uint32{0, 0}); err != nil {
t.Errorf("Unflag(nuller) = %v", err)
}
msgs, _, err := client.Since(0, 25)
if err != nil {
t.Fatal(err)
}
if len(msgs) != 1 {
t.Errorf("mailen mistede sit flag, uden at nogen havde adresseret den")
}
}
+132 -13
View File
@@ -37,6 +37,23 @@ type Runner struct {
// startable from main.
SyncGmail func(ctx context.Context, user *store.User) (int, error)
// SyncMailbox reads one IMAP mailbox. Supplied by the HTTP layer for the same
// reason as SyncGmail: it owns turning a message into a task.
SyncMailbox func(ctx context.Context, user *store.User, m *store.Mailbox) (int, error)
// SyncCalendars refreshes one person's connected calendars. Supplied the same
// way and for the same reason: the HTTP layer owns the token refresh.
SyncCalendars func(ctx context.Context, user *store.User) (int, error)
// Push delivers a notification to somebody's devices. Supplied by the HTTP
// layer, which owns the VAPID keys and the subscription list.
Push func(userID, title, body, projectID string)
// SendBeacon reports this installation to the collector, if it is switched on
// and a day has passed. Supplied by the HTTP layer for the same reason as the
// syncs: that is where the instance settings and the version live.
SendBeacon func(ctx context.Context) error
wg sync.WaitGroup
}
@@ -60,7 +77,18 @@ func (r *Runner) Start(ctx context.Context) {
// need a public webhook and a Cloud Pub/Sub topic, which is a lot of Google
// account for a to-do app. Ten minutes is fast enough for something somebody
// starred and slow enough not to look like abuse.
r.every(ctx, "gmail", 10*time.Minute, r.syncGmailAccounts)
r.every(ctx, "mailboxes", 10*time.Minute, r.syncMailboxes)
// Fifteen minutes rather than ten. A calendar is not a mailbox: nothing here
// becomes a task, so being a quarter of an hour behind on a meeting next
// Tuesday costs nothing — and the "fetch now" button covers the one case where
// it matters, which is somebody who has just moved something and is looking at
// the grid to check. A person who has just connected does not wait for this
// either; connecting fetches the list, and the first sync is one click away.
r.every(ctx, "calendars", 15*time.Minute, r.syncCalendars)
// Checked hourly, sent at most daily — the same shape as the backup, and for
// the same reason: a container asleep at the scheduled minute would otherwise
// skip the day. SendBeacon decides whether a day has passed.
r.every(ctx, "beacon", time.Hour, r.sendBeacon)
}
func (r *Runner) Wait() { r.wg.Wait() }
@@ -95,6 +123,17 @@ func (r *Runner) every(ctx context.Context, name string, interval time.Duration,
}()
}
// sendBeacon reports that this installation exists, at most once a day.
//
// Every decision about what is sent, and why this is on by default rather than
// compulsory, lives in internal/beacon. This is only the clock.
func (r *Runner) sendBeacon(ctx context.Context) error {
if r.SendBeacon == nil {
return nil
}
return r.SendBeacon(ctx)
}
// --- reminders ------------------------------------------------------------------
// sendDueReminders delivers everything that has come due since the last pass.
@@ -122,6 +161,15 @@ func (r *Runner) sendDueReminders(ctx context.Context) error {
"due": rem.RemindAt.Format(time.RFC3339),
})
// And to the devices that asked for it. This was missing: reminders went out
// in-app and by mail, so somebody with the tab closed and no mail configured
// got a notification that never arrived — which is the case a reminder is
// for. Notifications from other people's actions did push; only the timed
// ones did not, and the interface said "on for this device" either way.
if r.Push != nil {
r.Push(rem.UserID, rem.TaskContent, reminderBody(rem.RemindAt), rem.ProjectID)
}
if r.mail.Configured() && rem.UserEmail != "" {
link := r.cfg.BaseURL + "/projekt/" + rem.ProjectID
if err := r.mail.SendReminder(ctx, rem.UserEmail, rem.UserName, rem.TaskContent, link); err != nil {
@@ -135,6 +183,12 @@ func (r *Runner) sendDueReminders(ctx context.Context) error {
return nil
}
// reminderBody is the line under the task's title. The time rather than the date:
// a reminder arrives when it is due, so the day is today and saying so is noise.
func reminderBody(at time.Time) string {
return at.Format("15:04")
}
// --- backups ---------------------------------------------------------------------
const (
@@ -247,26 +301,91 @@ func rotateBackups(cfg *config.Config, log *slog.Logger) error {
// --- gmail ---------------------------------------------------------------------------
// syncGmailAccounts polls every connected mailbox.
// syncMailboxes polls every connected mailbox, Gmail and IMAP alike.
//
// One at a time, and a failure on one account does not stop the others: a revoked
// token or an expired refresh is a problem for that person alone.
func (r *Runner) syncGmailAccounts(ctx context.Context) error {
if r.SyncGmail == nil {
// One sweep rather than two, now that both live in the same table. Nobody is
// waiting, so no budget of its own — the request handler has one because a person
// is watching a spinner. A host that has gone quiet costs this run its own dial
// timeout and nothing else: the loop moves on, per mailbox and per person, so one
// bad password does not stop everybody else's mail.
func (r *Runner) syncMailboxes(ctx context.Context) error {
if r.SyncMailbox == nil {
return nil
}
users, err := r.db.UsersWithGmail(ctx)
userIDs, err := r.db.UsersWithMailboxes(ctx)
if err != nil {
return err
}
for _, user := range users {
created, err := r.SyncGmail(ctx, &user)
if err != nil {
r.log.Warn("gmail sync", "err", err, "user", user.ID)
for _, id := range userIDs {
user, err := r.db.UserByID(ctx, id)
if err != nil || user == nil {
continue
}
if created > 0 {
r.log.Info("tasks created from gmail", "user", user.ID, "count", created)
boxes, err := r.db.Mailboxes(ctx, id)
if err != nil {
r.log.Warn("list mailboxes", "err", err, "user", id)
continue
}
for i := range boxes {
var created int
var err error
// Gmail is fetched over HTTP with a token, IMAP over a connection with a
// password. Which one is a property of the row, not of the loop.
if boxes[i].Kind == "gmail" {
if r.SyncGmail == nil {
continue
}
created, err = r.SyncGmail(ctx, user)
} else {
created, err = r.SyncMailbox(ctx, user, &boxes[i])
}
if err != nil {
r.log.Warn("mailbox sync", "err", err, "user", id,
"kind", boxes[i].Kind, "mailbox", boxes[i].ID)
continue
}
if created > 0 {
r.log.Info("tasks created from a mailbox", "user", id,
"kind", boxes[i].Kind, "count", created)
}
}
}
return nil
}
// --- calendars ------------------------------------------------------------------------
// syncCalendars refreshes everybody's connected calendars.
//
// Its own sweep rather than a branch inside the mailbox one. They share nothing
// past the word "poll": that loop asks each mailbox how far it has read and makes
// tasks, this one replaces a window of somebody else's events. Folding them
// together would mean one interval for two questions that do not want the same
// answer, and a Google that is down would count against a mailbox that is not.
//
// Nobody is waiting, so no budget of its own — the request handler has one because
// a person is watching a spinner. One account that has gone quiet costs this run
// its own timeout and nothing else: the loop moves on per person.
func (r *Runner) syncCalendars(ctx context.Context) error {
if r.SyncCalendars == nil {
return nil
}
userIDs, err := r.db.UsersWithCalendars(ctx)
if err != nil {
return err
}
for _, id := range userIDs {
user, err := r.db.UserByID(ctx, id)
if err != nil || user == nil {
continue
}
written, err := r.SyncCalendars(ctx, user)
if err != nil {
r.log.Warn("calendar sync", "err", err, "user", id)
continue
}
if written > 0 {
r.log.Info("calendar refreshed", "user", id, "events", written)
}
}
return nil
+56
View File
@@ -276,3 +276,59 @@ func backupFiles(t *testing.T, cfg *config.Config) []string {
}
return out
}
// A reminder that nobody receives is not a reminder.
//
// The job sent them in-app and by mail. Somebody with the tab closed and no mail
// server configured — which is the setup the docker one-liner produces — got
// nothing, while the settings page said notifications were on for the device.
// The push half was simply never wired: notifications from other people's actions
// went out, only the timed ones did not.
func TestADueReminderReachesTheDevices(t *testing.T) {
runner, db, _ := newRunner(t)
user := seedUser(t, db)
type sent struct{ userID, title, projectID string }
var got []sent
runner.Push = func(userID, title, body, projectID string) {
got = append(got, sent{userID, title, projectID})
}
inbox, err := db.InboxID(t.Context(), user.ID)
if err != nil {
t.Fatal(err)
}
task := &store.Task{ProjectID: inbox, Content: "ring til Anders", CreatedBy: user.ID}
if err := db.CreateTask(t.Context(), task, nil); err != nil {
t.Fatal(err)
}
// A minute ago, so it is due rather than upcoming.
at := time.Now().Add(-time.Minute)
if _, err := db.CreateReminder(t.Context(), task.ID, user.ID, &at, nil); err != nil {
t.Fatal(err)
}
if err := runner.sendDueReminders(t.Context()); err != nil {
t.Fatal(err)
}
if len(got) != 1 {
t.Fatalf("%d reminders reached the devices, want 1", len(got))
}
if got[0].userID != user.ID {
t.Errorf("it went to %q", got[0].userID)
}
if got[0].title != "ring til Anders" {
t.Errorf("the title is %q, not the task", got[0].title)
}
// And once. A reminder that arrives on every sweep is worse than one that does
// not arrive at all.
got = nil
if err := runner.sendDueReminders(t.Context()); err != nil {
t.Fatal(err)
}
if len(got) != 0 {
t.Errorf("the same reminder went out %d more times", len(got))
}
}
+136
View File
@@ -0,0 +1,136 @@
// Package safedial refuses to open a connection to this machine or to the
// network it sits on.
//
// The problem it solves is the same one three times: somewhere a user supplies a
// URL or a host, and the server fetches it. An AI provider's base address, a push
// endpoint, an IMAP host, a beacon collector. Each of them is a feature, and each
// of them is also a way to ask the server to reach something the caller cannot —
// the panel next door, a metadata endpoint, a database on the same bridge — and
// to report back whether it answered. On a homelab that is the whole point of the
// box being where it is.
//
// # Why the check is on the address and not on the URL
//
// Parsing the URL and refusing "localhost" or "127.0.0.1" is the obvious version
// and it does not work: a name resolves to whatever its owner says, so
// `evil.example.com` can answer 127.0.0.1 — and can answer differently the second
// time, after a parse-time check has passed. That is DNS rebinding, and the only
// place it can be caught is after resolution and before the connection, which is
// exactly where net.Dialer's Control hook runs.
//
// # What is refused
//
// Loopback, link-local (including 169.254.169.254, which is where cloud metadata
// lives), the private ranges, unique-local IPv6, and the unspecified address.
// Everything else is allowed: this is not an allowlist of the internet, it is a
// wall around the inside.
package safedial
import (
"fmt"
"net"
"net/http"
"net/url"
"syscall"
"time"
)
// ErrPrivate is returned when a connection was refused for where it pointed. It
// deliberately does not say which address it resolved to: the answer is itself
// the thing an attacker is fishing for.
type ErrPrivate struct{ Host string }
func (e *ErrPrivate) Error() string {
return fmt.Sprintf("refusing to connect to %s: it resolves to an address on this machine or its private network", e.Host)
}
// Blocked reports whether an address must not be dialled.
func Blocked(ip net.IP) bool {
return ip == nil ||
ip.IsLoopback() ||
ip.IsUnspecified() ||
ip.IsLinkLocalUnicast() ||
ip.IsLinkLocalMulticast() ||
ip.IsInterfaceLocalMulticast() ||
ip.IsMulticast() ||
ip.IsPrivate()
}
// Control is a net.Dialer Control function that refuses a private destination.
//
// It runs after the name has been resolved and before the socket is connected,
// which is the only moment where the address that will actually be used is known.
func Control(network, address string, _ syscall.RawConn) error {
host, _, err := net.SplitHostPort(address)
if err != nil {
return err
}
ip := net.ParseIP(host)
if Blocked(ip) {
return &ErrPrivate{Host: host}
}
return nil
}
// Dialer is a net.Dialer that will not reach inside.
func Dialer(timeout time.Duration) *net.Dialer {
return &net.Dialer{Timeout: timeout, Control: Control}
}
// Client is an http.Client for fetching something a user named.
//
// Redirects are followed but capped, and every hop goes through the same dialer —
// a redirect to 127.0.0.1 is the oldest way around a check that only looked at the
// URL somebody typed.
func Client(timeout time.Duration) *http.Client {
return &http.Client{
Timeout: timeout,
Transport: &http.Transport{
DialContext: Dialer(timeout).DialContext,
TLSHandshakeTimeout: timeout,
ResponseHeaderTimeout: timeout,
},
CheckRedirect: func(req *http.Request, via []*http.Request) error {
if len(via) >= 5 {
return fmt.Errorf("too many redirects")
}
return nil
},
}
}
// AllowPrivate is the same client without the wall, for a destination the
// operator configured rather than a user — the panel, and nothing else. Named so
// that using it is a decision somebody wrote down.
func AllowPrivate(timeout time.Duration) *http.Client {
return &http.Client{Timeout: timeout}
}
// CheckURL reports why an address must not be fetched, or "" if it may be.
//
// A convenience for saying no at the moment somebody types an address, rather
// than later when a request quietly fails. It is NOT the protection — a name can
// resolve one way now and another way when it is dialled, which is why Control
// exists and runs every time. Anything this rejects is certainly wrong; anything
// it accepts is merely not obviously wrong.
func CheckURL(raw string) string {
u, err := url.Parse(raw)
if err != nil || u.Host == "" {
return "that is not a URL"
}
host := u.Hostname()
if ip := net.ParseIP(host); ip != nil && Blocked(ip) {
return "that address is on this machine or its private network"
}
// A bare name is looked up once so an obvious "localhost" is caught here. A
// failure to resolve is not an error: the host may simply not exist yet, and
// refusing to save an address because DNS was slow would be its own bug.
if ips, err := net.LookupIP(host); err == nil {
for _, ip := range ips {
if Blocked(ip) {
return "that address is on this machine or its private network"
}
}
}
return ""
}
+55
View File
@@ -0,0 +1,55 @@
package safedial
import (
"net"
"net/http"
"net/http/httptest"
"testing"
"time"
)
func TestBlockedCoversTheInside(t *testing.T) {
blocked := []string{
"127.0.0.1", "::1", "0.0.0.0",
// Skyens metadata-endepunkt. Den ene adresse, enhver SSRF sigter efter.
"169.254.169.254",
"10.0.0.5", "172.16.0.1", "192.168.1.10", "fd00::1", "fe80::1",
}
for _, s := range blocked {
if !Blocked(net.ParseIP(s)) {
t.Errorf("Blocked(%s) = false", s)
}
}
for _, s := range []string{"93.184.216.34", "8.8.8.8", "2606:2800:220:1:248:1893:25c8:1946"} {
if Blocked(net.ParseIP(s)) {
t.Errorf("Blocked(%s) = true, men det er en almindelig adresse", s)
}
}
}
// En httptest-server lytter på loopback, hvilket gør den til den perfekte prøve:
// den findes, den svarer, og den må ikke kunne nås.
func TestClientRefusesLoopback(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
t.Error("forbindelsen blev oprettet; den skulle være afvist")
}))
defer srv.Close()
_, err := Client(3 * time.Second).Get(srv.URL)
if err == nil {
t.Fatal("en forespørgsel til loopback skal afvises")
}
}
func TestAllowPrivateStillReaches(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusNoContent)
}))
defer srv.Close()
resp, err := AllowPrivate(3 * time.Second).Get(srv.URL)
if err != nil {
t.Fatalf("den skal kunne nå det, operatøren selv har peget på: %v", err)
}
resp.Body.Close()
}
+138
View File
@@ -0,0 +1,138 @@
package store
import (
"context"
"time"
)
// The collector's side of the beacon: what has reported in, and how many.
//
// Every method here is about counting installations, never about identifying one.
// There is no lookup by anything but the id an installation made up for itself,
// and there is nothing to join it against.
// BeaconStats is the whole of what the collector can say.
type BeaconStats struct {
Total int `json:"total"`
Active7d int `json:"active_7d"`
Active30d int `json:"active_30d"`
ByVersion map[string]int `json:"by_version"`
LastPingAt *time.Time `json:"last_ping_at,omitempty"`
}
// maxBeaconInstalls bounds the table.
//
// The receiving endpoint is unauthenticated — it has to be, since the caller is a
// stranger's installation — and it inserts a row keyed on an id the caller made
// up. Anybody can therefore mint ids and grow this table until the volume the
// database lives on is full. A ceiling turns that from a disk-fill into a number
// that is merely wrong, which is the difference between an outage and an
// annoyance. A hundred thousand is far above any plausible real count and far
// below anything that matters on disk.
const maxBeaconInstalls = 100_000
// RecordBeacon writes down that an installation is alive, and on which version.
//
// `first_seen` is kept from the original row on purpose: it is the only thing
// that distinguishes an installation that has been running for a year from one
// that appeared this morning, and it costs nothing to keep.
func (db *DB) RecordBeacon(ctx context.Context, instanceID, version string) error {
now := time.Now().Unix()
// An installation already known keeps reporting whatever the ceiling says: the
// cap must never stop counting the real ones, only stop new rows from being
// invented without limit.
var known int
if err := db.QueryRowContext(ctx,
`SELECT count(*) FROM beacon_installs WHERE instance_id = ?`, instanceID).Scan(&known); err != nil {
return err
}
if known == 0 {
var total int
if err := db.QueryRowContext(ctx,
`SELECT count(*) FROM beacon_installs`).Scan(&total); err != nil {
return err
}
if total >= maxBeaconInstalls {
// Not an error to the caller. Their installation did nothing wrong, and
// there is nothing for them to do about it.
return nil
}
}
_, err := db.ExecContext(ctx, `
INSERT INTO beacon_installs (instance_id, version, first_seen, last_seen)
VALUES (?, ?, ?, ?)
ON CONFLICT (instance_id) DO UPDATE SET
version = excluded.version,
last_seen = excluded.last_seen`,
instanceID, version, now, now)
return err
}
// BeaconCount is how many installations have reported within the window.
//
// A window rather than a running total, and that is not a detail. A total only
// ever climbs: every instance anybody ever started for ten minutes stays in it
// forever, so the number stops meaning "how many people use this" and starts
// meaning "how many people have ever tried it". The question being asked is the
// first one.
func (db *DB) BeaconCount(ctx context.Context, within time.Duration) (int, error) {
var n int
err := db.QueryRowContext(ctx,
`SELECT count(*) FROM beacon_installs WHERE last_seen >= ?`,
time.Now().Add(-within).Unix()).Scan(&n)
return n, err
}
// BeaconSummary is everything the settings page shows.
func (db *DB) BeaconSummary(ctx context.Context) (BeaconStats, error) {
out := BeaconStats{ByVersion: map[string]int{}}
if err := db.QueryRowContext(ctx,
`SELECT count(*) FROM beacon_installs`).Scan(&out.Total); err != nil {
return out, err
}
var err error
if out.Active7d, err = db.BeaconCount(ctx, 7*24*time.Hour); err != nil {
return out, err
}
if out.Active30d, err = db.BeaconCount(ctx, 30*24*time.Hour); err != nil {
return out, err
}
// Versions of the ones still alive, not of everything ever seen: an instance
// that stopped reporting two years ago should not still be counted as running
// the version it was on when it stopped.
rows, err := db.QueryContext(ctx, `
SELECT version, count(*) FROM beacon_installs
WHERE last_seen >= ? AND version <> ''
GROUP BY version`, time.Now().Add(-30*24*time.Hour).Unix())
if err != nil {
return out, err
}
defer rows.Close()
for rows.Next() {
var v string
var n int
if err := rows.Scan(&v, &n); err != nil {
return out, err
}
out.ByVersion[v] = n
}
if err := rows.Err(); err != nil {
return out, err
}
var last *int64
if err := db.QueryRowContext(ctx,
`SELECT max(last_seen) FROM beacon_installs`).Scan(&last); err != nil {
return out, err
}
if last != nil {
t := time.Unix(*last, 0)
out.LastPingAt = &t
}
return out, nil
}
+380
View File
@@ -0,0 +1,380 @@
package store
import (
"context"
"database/sql"
"fmt"
"time"
)
// CalendarAccount is one connected account belonging to one person.
type CalendarAccount struct {
ID string `json:"id"`
UserID string `json:"-"`
Provider string `json:"provider"`
Account string `json:"account,omitempty"`
RefreshToken string `json:"-"` // never leaves the server
AccessToken string `json:"-"`
ExpiresAt time.Time `json:"-"`
LastSyncAt time.Time `json:"last_sync_at,omitempty"`
CreatedAt time.Time `json:"created_at"`
}
// Calendar is one calendar inside an account, shown or not.
type Calendar struct {
ID string `json:"id"`
AccountID string `json:"-"`
RemoteID string `json:"remote_id"`
Name string `json:"name"`
Colour string `json:"colour,omitempty"`
TimeZone string `json:"time_zone,omitempty"`
Primary bool `json:"primary"`
// Writable says the connected account could write to it. Nothing does — the
// scope is read-only — and it is stored so the day something can, the interface
// knows which calendars could even be offered.
Writable bool `json:"writable"`
Shown bool `json:"shown"`
SortOrder int `json:"-"`
}
// CalendarEvent is one occurrence, in the shape a grid asks about: days at both
// ends, inclusive.
type CalendarEvent struct {
ID string `json:"id"`
CalendarID string `json:"calendar_id"`
RemoteID string `json:"-"`
Summary string `json:"summary"`
StartsAt string `json:"starts_at,omitempty"`
EndsAt string `json:"ends_at,omitempty"`
StartDay string `json:"start_day"`
EndDay string `json:"end_day"`
AllDay bool `json:"all_day"`
Location string `json:"location,omitempty"`
URL string `json:"url,omitempty"`
// CalendarName and Colour ride along with the event rather than being looked up
// per row by the caller. Same reasoning as the assignee names on a task list:
// the chip has to be drawn in the calendar's own colour, and a second request
// per event to learn it is a request per event.
CalendarName string `json:"calendar_name,omitempty"`
Colour string `json:"colour,omitempty"`
}
const calendarAccountColumns = `id, user_id, provider, account, refresh_token,
access_token, expires_at, last_sync_at, created_at`
// CalendarAccountFor returns a person's connection to one provider, or nil.
func (db *DB) CalendarAccountFor(ctx context.Context, userID, provider string) (*CalendarAccount, error) {
row := db.QueryRowContext(ctx, `SELECT `+calendarAccountColumns+`
FROM calendar_accounts WHERE user_id = ? AND provider = ?`, userID, provider)
a, err := db.scanCalendarAccount(row)
if err == sql.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, err
}
return &a, nil
}
// SaveCalendarAccount writes one, sealing the two columns that must not travel
// with a backup. Called by hand rather than by the settings tables' own seal,
// which does not reach this table.
func (db *DB) SaveCalendarAccount(ctx context.Context, a *CalendarAccount) error {
if a.ID == "" {
a.ID = NewID()
}
if a.Provider == "" {
a.Provider = "google"
}
if a.CreatedAt.IsZero() {
a.CreatedAt = time.Now()
}
refresh, err := db.sealValue(a.RefreshToken)
if err != nil {
return fmt.Errorf("seal refresh_token: %w", err)
}
access, err := db.sealValue(a.AccessToken)
if err != nil {
return fmt.Errorf("seal access_token: %w", err)
}
// The conflict is on the pair, not on the id: a reconnect builds a fresh
// struct with no id in it, and matching on the id would insert a second row
// that the unique index then refuses — which reads to the person as "connecting
// failed" when what happened is that it already had.
_, err = db.ExecContext(ctx, `
INSERT INTO calendar_accounts (`+calendarAccountColumns+`)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (user_id, provider) DO UPDATE SET
account = excluded.account,
refresh_token = excluded.refresh_token,
access_token = excluded.access_token,
expires_at = excluded.expires_at,
last_sync_at = excluded.last_sync_at`,
a.ID, a.UserID, a.Provider, a.Account, refresh, access,
unixOrZero(a.ExpiresAt), unixOrZero(a.LastSyncAt), a.CreatedAt.Unix())
if err != nil {
return err
}
// The row that is actually there wins. On a reconnect the insert above updated
// an existing row, so the id in hand is one nothing points at — and the
// calendars are hung off the id, so writing them against it would attach them
// to nothing.
return db.QueryRowContext(ctx,
`SELECT id FROM calendar_accounts WHERE user_id = ? AND provider = ?`,
a.UserID, a.Provider).Scan(&a.ID)
}
// MarkCalendarSynced records how far a run got. Written separately from the rest
// so a sync never rewrites the credentials it is holding in memory.
func (db *DB) MarkCalendarSynced(ctx context.Context, accountID string, at time.Time) error {
_, err := db.ExecContext(ctx,
`UPDATE calendar_accounts SET last_sync_at = ? WHERE id = ?`, at.Unix(), accountID)
return err
}
// DeleteCalendarAccount disconnects one. The calendars and their events go with it
// — unlike the tasks a mailbox made, none of this is the person's own work; it is
// a copy of somebody else's calendar and it has no meaning without the account.
func (db *DB) DeleteCalendarAccount(ctx context.Context, userID, id string) error {
_, err := db.ExecContext(ctx,
`DELETE FROM calendar_accounts WHERE id = ? AND user_id = ?`, id, userID)
return err
}
// UsersWithCalendars returns everybody the sweep has to visit.
func (db *DB) UsersWithCalendars(ctx context.Context) ([]string, error) {
rows, err := db.QueryContext(ctx, `SELECT DISTINCT user_id FROM calendar_accounts`)
if err != nil {
return nil, err
}
defer rows.Close()
var ids []string
for rows.Next() {
var id string
if err := rows.Scan(&id); err != nil {
return nil, err
}
ids = append(ids, id)
}
return ids, rows.Err()
}
// Calendars returns what one account holds, in the order the provider listed them.
func (db *DB) Calendars(ctx context.Context, accountID string) ([]Calendar, error) {
rows, err := db.QueryContext(ctx, `
SELECT id, account_id, remote_id, name, colour, time_zone, primary_one,
writable, shown, sort_order
FROM calendars WHERE account_id = ? ORDER BY sort_order, name`, accountID)
if err != nil {
return nil, err
}
defer rows.Close()
var out []Calendar
for rows.Next() {
var c Calendar
if err := rows.Scan(&c.ID, &c.AccountID, &c.RemoteID, &c.Name, &c.Colour,
&c.TimeZone, &c.Primary, &c.Writable, &c.Shown, &c.SortOrder); err != nil {
return nil, err
}
out = append(out, c)
}
return out, rows.Err()
}
// ReplaceCalendars writes the list the provider just gave, keeping what the person
// chose.
//
// The choice survives because it is not in the list: Google says which calendars
// exist and what they are called, and only verdande knows which of them somebody
// wants to look at. Merged rather than deleted and rewritten, so ticking a calendar
// on does not come undone the next time the sweep runs.
//
// A calendar that has disappeared from the account is removed, and its events go
// with it — a calendar somebody was unshared from should stop being drawn, not
// stay behind as a frozen copy of what it held on the last good day.
func (db *DB) ReplaceCalendars(ctx context.Context, accountID string, list []Calendar) error {
return db.Tx(ctx, func(tx *sql.Tx) error {
keep := map[string]bool{}
for i, c := range list {
keep[c.RemoteID] = true
// The insert cannot carry `shown`: on conflict it would overwrite the
// choice with the default. A new calendar starts hidden — an account
// with a dozen shared calendars in it should not fill somebody's grid
// the moment they connect it.
if _, err := tx.ExecContext(ctx, `
INSERT INTO calendars
(id, account_id, remote_id, name, colour, time_zone, primary_one,
writable, shown, sort_order)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (account_id, remote_id) DO UPDATE SET
name = excluded.name,
colour = excluded.colour,
time_zone = excluded.time_zone,
primary_one = excluded.primary_one,
writable = excluded.writable,
sort_order = excluded.sort_order`,
NewID(), accountID, c.RemoteID, c.Name, c.Colour, c.TimeZone,
c.Primary, c.Writable, c.Shown, i); err != nil {
return err
}
}
rows, err := tx.QueryContext(ctx,
`SELECT id, remote_id FROM calendars WHERE account_id = ?`, accountID)
if err != nil {
return err
}
var gone []string
for rows.Next() {
var id, remote string
if err := rows.Scan(&id, &remote); err != nil {
rows.Close()
return err
}
if !keep[remote] {
gone = append(gone, id)
}
}
// Closed before the next statement runs on this transaction. An open result
// set with a query behind it is the deadlock the pool comment in store.go
// exists to make slow rather than fatal.
rows.Close()
if err := rows.Err(); err != nil {
return err
}
for _, id := range gone {
if _, err := tx.ExecContext(ctx, `DELETE FROM calendars WHERE id = ?`, id); err != nil {
return err
}
}
return nil
})
}
// ShowCalendars sets which of an account's calendars are drawn.
//
// The whole set in one write rather than one flip at a time, for the reason the
// projects' order is written the same way: a person has a handful of calendars, the
// question is "these ones", and a list that can only be built by several requests
// can land half applied.
func (db *DB) ShowCalendars(ctx context.Context, accountID string, shown []string) error {
return db.Tx(ctx, func(tx *sql.Tx) error {
if _, err := tx.ExecContext(ctx,
`UPDATE calendars SET shown = 0 WHERE account_id = ?`, accountID); err != nil {
return err
}
for _, id := range shown {
// Scoped in the SQL rather than checked first: an id read in one step
// and written in the next is a window, and the account is the whole of
// the permission here.
if _, err := tx.ExecContext(ctx,
`UPDATE calendars SET shown = 1 WHERE id = ? AND account_id = ?`,
id, accountID); err != nil {
return err
}
}
// Events of a calendar nobody is looking at are dead weight in a table the
// grid queries. They come back with the next sync if it is turned on again.
_, err := tx.ExecContext(ctx, `
DELETE FROM calendar_events WHERE calendar_id IN
(SELECT id FROM calendars WHERE account_id = ? AND shown = 0)`, accountID)
return err
})
}
// ReplaceCalendarEvents swaps one calendar's events inside a window.
//
// Replaced rather than merged, and no sync token. Google offers incremental sync,
// and it would save bytes verdande is not short of while costing the one thing it
// cannot afford: a token that has to be right, forever, for the grid not to be
// quietly wrong. A window that is deleted and rewritten cannot drift — an event
// somebody moved out of the window disappears because it is not in the answer, not
// because verdande remembered to delete it.
//
// Bounded by the window at both ends, so a sync of September does not throw away
// August.
func (db *DB) ReplaceCalendarEvents(ctx context.Context, calendarID, from, to string, events []CalendarEvent) error {
return db.Tx(ctx, func(tx *sql.Tx) error {
if _, err := tx.ExecContext(ctx, `
DELETE FROM calendar_events
WHERE calendar_id = ? AND end_day >= ? AND start_day <= ?`,
calendarID, from, to); err != nil {
return err
}
for _, e := range events {
if _, err := tx.ExecContext(ctx, `
INSERT INTO calendar_events
(id, calendar_id, remote_id, summary, starts_at, ends_at,
start_day, end_day, all_day, location, url)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
NewID(), calendarID, e.RemoteID, e.Summary, e.StartsAt, e.EndsAt,
e.StartDay, e.EndDay, e.AllDay, e.Location, e.URL); err != nil {
return err
}
}
return nil
})
}
// CalendarEvents returns everything one person can see between two days,
// inclusive at both ends.
//
// Scoped to the caller in the join rather than checked afterwards: the events of
// somebody else's calendar are somebody else's, and a filter applied after the read
// is a filter that can be forgotten.
func (db *DB) CalendarEvents(ctx context.Context, userID, from, to string) ([]CalendarEvent, error) {
rows, err := db.QueryContext(ctx, `
SELECT e.id, e.calendar_id, e.remote_id, e.summary, e.starts_at, e.ends_at,
e.start_day, e.end_day, e.all_day, e.location, e.url, c.name, c.colour
FROM calendar_events e
JOIN calendars c ON c.id = e.calendar_id
JOIN calendar_accounts a ON a.id = c.account_id
WHERE a.user_id = ? AND c.shown = 1 AND e.end_day >= ? AND e.start_day <= ?
ORDER BY e.start_day, e.all_day DESC, e.starts_at`, userID, from, to)
if err != nil {
return nil, err
}
defer rows.Close()
var out []CalendarEvent
for rows.Next() {
var e CalendarEvent
if err := rows.Scan(&e.ID, &e.CalendarID, &e.RemoteID, &e.Summary, &e.StartsAt,
&e.EndsAt, &e.StartDay, &e.EndDay, &e.AllDay, &e.Location, &e.URL,
&e.CalendarName, &e.Colour); err != nil {
return nil, err
}
out = append(out, e)
}
return out, rows.Err()
}
func (db *DB) scanCalendarAccount(row scanner) (CalendarAccount, error) {
var a CalendarAccount
var expires, lastSync, created int64
err := row.Scan(&a.ID, &a.UserID, &a.Provider, &a.Account, &a.RefreshToken,
&a.AccessToken, &expires, &lastSync, &created)
if err != nil {
return a, err
}
for _, field := range []*string{&a.RefreshToken, &a.AccessToken} {
plain, err := db.unsealValue(*field)
if err != nil {
return a, err
}
*field = plain
}
a.ExpiresAt = timeOrZero(expires)
a.LastSyncAt = timeOrZero(lastSync)
a.CreatedAt = time.Unix(created, 0)
return a, nil
}
+358
View File
@@ -0,0 +1,358 @@
package store
import (
"context"
"strings"
"testing"
"time"
)
func connectedAccount(t *testing.T, db *DB, userID string) *CalendarAccount {
t.Helper()
a := &CalendarAccount{
UserID: userID, Provider: "google", Account: "[email protected]",
RefreshToken: "1//0-a-real-looking-refresh-token", AccessToken: "ya29.access",
ExpiresAt: time.Now().Add(time.Hour),
}
if err := db.SaveCalendarAccount(context.Background(), a); err != nil {
t.Fatal(err)
}
return a
}
// The same claim the mailbox tokens have to meet: what is written must not be
// readable in the file a backup copies.
func TestCalendarTokensAreNotStoredInTheClear(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
a := connectedAccount(t, db, userID)
var refresh, access string
if err := db.QueryRowContext(ctx,
`SELECT refresh_token, access_token FROM calendar_accounts WHERE id = ?`,
a.ID).Scan(&refresh, &access); err != nil {
t.Fatal(err)
}
if strings.Contains(refresh, "real-looking") || strings.Contains(access, "ya29") {
t.Fatalf("a token is in the row as written: %q %q", refresh, access)
}
// The address stays readable, which is the reason for sealing named columns
// rather than the whole row.
var account string
if err := db.QueryRowContext(ctx,
`SELECT account FROM calendar_accounts WHERE id = ?`, a.ID).Scan(&account); err != nil {
t.Fatal(err)
}
if account != "[email protected]" {
t.Errorf("the address was sealed too: %q", account)
}
back, err := db.CalendarAccountFor(ctx, userID, "google")
if err != nil {
t.Fatal(err)
}
if back.RefreshToken != a.RefreshToken || back.AccessToken != a.AccessToken {
t.Errorf("the tokens did not survive the round trip: %+v", back)
}
}
// Connecting again must replace what was there, not fail. It is the same OAuth
// flow run a second time, and the person's word for it is "connect", not "add".
func TestConnectingAgainReplacesTheAccount(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
first := connectedAccount(t, db, userID)
second := &CalendarAccount{
UserID: userID, Provider: "google", Account: "[email protected]",
RefreshToken: "1//0-fresh", AccessToken: "ya29.fresh",
}
if err := db.SaveCalendarAccount(ctx, second); err != nil {
t.Fatalf("connecting a second time failed: %v", err)
}
// And the id in hand is the row that is actually there. The calendars hang off
// it, so an id nothing points at would attach them to nothing.
if second.ID != first.ID {
t.Errorf("the second connect came back with id %q, want the existing %q",
second.ID, first.ID)
}
back, err := db.CalendarAccountFor(ctx, userID, "google")
if err != nil {
t.Fatal(err)
}
if back.RefreshToken != "1//0-fresh" {
t.Errorf("the token came back as %q", back.RefreshToken)
}
}
// Google says which calendars exist; only verdande knows which ones somebody wants
// to look at. A refresh that came back with the same list must not undo the choice.
func TestRefreshingTheListKeepsWhatWasChosen(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
a := connectedAccount(t, db, userID)
list := []Calendar{
{RemoteID: "primary", Name: "[email protected]", Colour: "#3f51b5", Primary: true},
{RemoteID: "fam", Name: "Familien", Colour: "#0b8043"},
{RemoteID: "old", Name: "Et delt"},
}
if err := db.ReplaceCalendars(ctx, a.ID, list); err != nil {
t.Fatal(err)
}
// Nothing is shown to begin with: an account with a dozen shared calendars in
// it should not fill somebody's grid the moment they connect it.
stored, err := db.Calendars(ctx, a.ID)
if err != nil {
t.Fatal(err)
}
if len(stored) != 3 {
t.Fatalf("got %d calendars, want 3", len(stored))
}
for _, c := range stored {
if c.Shown {
t.Errorf("%q was shown without anybody asking for it", c.Name)
}
}
var famID string
for _, c := range stored {
if c.RemoteID == "fam" {
famID = c.ID
}
}
if err := db.ShowCalendars(ctx, a.ID, []string{famID}); err != nil {
t.Fatal(err)
}
// The sweep runs again, and one calendar has been renamed and one has gone.
if err := db.ReplaceCalendars(ctx, a.ID, []Calendar{
{RemoteID: "primary", Name: "[email protected]", Colour: "#3f51b5", Primary: true},
{RemoteID: "fam", Name: "Familien og hunden", Colour: "#0b8043"},
}); err != nil {
t.Fatal(err)
}
stored, err = db.Calendars(ctx, a.ID)
if err != nil {
t.Fatal(err)
}
if len(stored) != 2 {
t.Fatalf("got %d calendars; one that has left the account should stop being drawn",
len(stored))
}
for _, c := range stored {
switch c.RemoteID {
case "fam":
if !c.Shown {
t.Error("the choice was undone by a refresh of the list")
}
if c.Name != "Familien og hunden" {
t.Errorf("the new name did not arrive: %q", c.Name)
}
if c.ID != famID {
t.Error("the row was rebuilt rather than updated; its id is what was chosen")
}
case "primary":
if c.Shown {
t.Error("a calendar nobody chose became shown")
}
}
}
}
// A window is replaced, not merged. An event moved out of it has to disappear
// because it is absent from the answer — and the months on either side must not go
// with it.
func TestReplacingAWindowLeavesTheMonthsAroundItAlone(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
a := connectedAccount(t, db, userID)
if err := db.ReplaceCalendars(ctx, a.ID, []Calendar{{RemoteID: "primary", Name: "Min"}}); err != nil {
t.Fatal(err)
}
stored, _ := db.Calendars(ctx, a.ID)
cal := stored[0]
if err := db.ShowCalendars(ctx, a.ID, []string{cal.ID}); err != nil {
t.Fatal(err)
}
july := CalendarEvent{RemoteID: "j", Summary: "Juli", StartDay: "2026-07-15", EndDay: "2026-07-15"}
august := CalendarEvent{RemoteID: "a", Summary: "August", StartDay: "2026-08-15", EndDay: "2026-08-15"}
if err := db.ReplaceCalendarEvents(ctx, cal.ID, "2026-07-01", "2026-07-31",
[]CalendarEvent{july}); err != nil {
t.Fatal(err)
}
if err := db.ReplaceCalendarEvents(ctx, cal.ID, "2026-08-01", "2026-08-31",
[]CalendarEvent{august}); err != nil {
t.Fatal(err)
}
got, err := db.CalendarEvents(ctx, userID, "2026-07-01", "2026-08-31")
if err != nil {
t.Fatal(err)
}
if len(got) != 2 {
t.Fatalf("got %d events, want both months: %+v", len(got), got)
}
// August is synced again and the meeting has been cancelled. July stays.
if err := db.ReplaceCalendarEvents(ctx, cal.ID, "2026-08-01", "2026-08-31", nil); err != nil {
t.Fatal(err)
}
got, err = db.CalendarEvents(ctx, userID, "2026-07-01", "2026-08-31")
if err != nil {
t.Fatal(err)
}
if len(got) != 1 || got[0].Summary != "Juli" {
t.Fatalf("got %+v; syncing August threw away July", got)
}
}
// The grid asks "which cells does this cover", so an event that straddles the edge
// of the question has to come back.
func TestAnEventThatStraddlesTheWindowIsFound(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
a := connectedAccount(t, db, userID)
if err := db.ReplaceCalendars(ctx, a.ID, []Calendar{{RemoteID: "c", Name: "Min", Colour: "#0b8043"}}); err != nil {
t.Fatal(err)
}
stored, _ := db.Calendars(ctx, a.ID)
if err := db.ShowCalendars(ctx, a.ID, []string{stored[0].ID}); err != nil {
t.Fatal(err)
}
if err := db.ReplaceCalendarEvents(ctx, stored[0].ID, "2026-08-01", "2026-09-30",
[]CalendarEvent{
{RemoteID: "trip", Summary: "Aarhus", StartDay: "2026-08-28", EndDay: "2026-09-02", AllDay: true},
}); err != nil {
t.Fatal(err)
}
// A week that contains neither end of it.
got, err := db.CalendarEvents(ctx, userID, "2026-08-31", "2026-09-01")
if err != nil {
t.Fatal(err)
}
if len(got) != 1 {
t.Fatalf("got %+v; an event covering the whole week was not found", got)
}
// The calendar's own colour rides along, or the chip has to ask for it per row.
if got[0].Colour != "#0b8043" || got[0].CalendarName != "Min" {
t.Errorf("the calendar did not come with the event: %+v", got[0])
}
}
// Somebody else's calendar is somebody else's.
func TestOnlyTheOwnersEventsComeBack(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
other := &User{Email: "[email protected]", Name: "Andreas"}
if err := db.CreateUser(ctx, other, "Indbakke"); err != nil {
t.Fatal(err)
}
theirs := &CalendarAccount{UserID: other.ID, Provider: "google", RefreshToken: "r"}
if err := db.SaveCalendarAccount(ctx, theirs); err != nil {
t.Fatal(err)
}
if err := db.ReplaceCalendars(ctx, theirs.ID, []Calendar{{RemoteID: "c", Name: "Deres"}}); err != nil {
t.Fatal(err)
}
stored, _ := db.Calendars(ctx, theirs.ID)
if err := db.ShowCalendars(ctx, theirs.ID, []string{stored[0].ID}); err != nil {
t.Fatal(err)
}
if err := db.ReplaceCalendarEvents(ctx, stored[0].ID, "2026-08-01", "2026-08-31",
[]CalendarEvent{{RemoteID: "x", Summary: "Privat", StartDay: "2026-08-10", EndDay: "2026-08-10"}}); err != nil {
t.Fatal(err)
}
got, err := db.CalendarEvents(ctx, userID, "2026-08-01", "2026-08-31")
if err != nil {
t.Fatal(err)
}
if len(got) != 0 {
t.Fatalf("somebody else's calendar came back: %+v", got)
}
}
// A calendar turned off stops being drawn, and stops costing the grid anything.
func TestTurningACalendarOffRemovesItsEvents(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
a := connectedAccount(t, db, userID)
if err := db.ReplaceCalendars(ctx, a.ID, []Calendar{{RemoteID: "c", Name: "Min"}}); err != nil {
t.Fatal(err)
}
stored, _ := db.Calendars(ctx, a.ID)
if err := db.ShowCalendars(ctx, a.ID, []string{stored[0].ID}); err != nil {
t.Fatal(err)
}
if err := db.ReplaceCalendarEvents(ctx, stored[0].ID, "2026-08-01", "2026-08-31",
[]CalendarEvent{{RemoteID: "x", Summary: "Møde", StartDay: "2026-08-10", EndDay: "2026-08-10"}}); err != nil {
t.Fatal(err)
}
if err := db.ShowCalendars(ctx, a.ID, nil); err != nil {
t.Fatal(err)
}
var count int
if err := db.QueryRowContext(ctx, `SELECT count(*) FROM calendar_events`).Scan(&count); err != nil {
t.Fatal(err)
}
if count != 0 {
t.Errorf("%d events of a calendar nobody is looking at are still in the table", count)
}
}
// Disconnecting takes the calendars and the events with it. Unlike the tasks a
// mailbox made, none of this is the person's own work — it is a copy of somebody
// else's calendar and it means nothing without the account.
func TestDisconnectingTakesEverythingWithIt(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
a := connectedAccount(t, db, userID)
if err := db.ReplaceCalendars(ctx, a.ID, []Calendar{{RemoteID: "c", Name: "Min"}}); err != nil {
t.Fatal(err)
}
stored, _ := db.Calendars(ctx, a.ID)
if err := db.ShowCalendars(ctx, a.ID, []string{stored[0].ID}); err != nil {
t.Fatal(err)
}
if err := db.ReplaceCalendarEvents(ctx, stored[0].ID, "2026-08-01", "2026-08-31",
[]CalendarEvent{{RemoteID: "x", Summary: "Møde", StartDay: "2026-08-10", EndDay: "2026-08-10"}}); err != nil {
t.Fatal(err)
}
if err := db.DeleteCalendarAccount(ctx, userID, a.ID); err != nil {
t.Fatal(err)
}
for _, table := range []string{"calendar_accounts", "calendars", "calendar_events"} {
var count int
if err := db.QueryRowContext(ctx, `SELECT count(*) FROM `+table).Scan(&count); err != nil {
t.Fatal(err)
}
if count != 0 {
t.Errorf("%s still holds %d rows after disconnecting", table, count)
}
}
}
// The sweep finds people by the presence of an account, so it has to keep working
// now that the credential in it is ciphertext.
func TestTheSweepFindsAConnectedCalendar(t *testing.T) {
db, userID := sealedStore(t)
connectedAccount(t, db, userID)
users, err := db.UsersWithCalendars(context.Background())
if err != nil {
t.Fatal(err)
}
if len(users) != 1 || users[0] != userID {
t.Fatalf("the sweep found %v", users)
}
}
+24 -8
View File
@@ -27,6 +27,7 @@ type Attachment struct {
// GroupID hangs the file on a project group rather than on any one task in
// it: the contract that governs all of "Arbejde", not a step in it.
GroupID string
NoteID string
Filename string
MimeType string
Size int64
@@ -157,17 +158,18 @@ func (db *DB) CreateAttachment(ctx context.Context, a *Attachment) error {
a.CreatedAt = time.Now().UTC()
_, err := db.ExecContext(ctx,
`INSERT INTO attachments (id, task_id, comment_id, group_id, filename, mime_type,
`INSERT INTO attachments (id, task_id, comment_id, group_id, note_id, filename, mime_type,
size, path, uploaded_by, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
a.ID, nullString(a.TaskID), nullString(a.CommentID), nullString(a.GroupID),
nullString(a.NoteID),
a.Filename, a.MimeType, a.Size, a.Path, a.UploadedBy, a.CreatedAt.Unix())
return err
}
func (db *DB) ListTaskAttachments(ctx context.Context, taskID string) ([]Attachment, error) {
rows, err := db.QueryContext(ctx,
`SELECT id, task_id, comment_id, group_id, filename, mime_type, size, path, uploaded_by, created_at
`SELECT id, task_id, comment_id, group_id, note_id, filename, mime_type, size, path, uploaded_by, created_at
FROM attachments WHERE task_id = ? ORDER BY created_at`, taskID)
if err != nil {
return nil, err
@@ -176,11 +178,24 @@ func (db *DB) ListTaskAttachments(ctx context.Context, taskID string) ([]Attachm
return scanAttachments(rows)
}
// ListNoteAttachments is the files inside one note — the pictures and scans that
// came with it, and anything added since.
func (db *DB) ListNoteAttachments(ctx context.Context, noteID string) ([]Attachment, error) {
rows, err := db.QueryContext(ctx,
`SELECT id, task_id, comment_id, group_id, note_id, filename, mime_type, size, path, uploaded_by, created_at
FROM attachments WHERE note_id = ? ORDER BY created_at`, noteID)
if err != nil {
return nil, err
}
defer rows.Close()
return scanAttachments(rows)
}
// ListGroupAttachments is the documents that belong to a whole group rather than
// to anything inside it.
func (db *DB) ListGroupAttachments(ctx context.Context, groupID string) ([]Attachment, error) {
rows, err := db.QueryContext(ctx,
`SELECT id, task_id, comment_id, group_id, filename, mime_type, size, path, uploaded_by, created_at
`SELECT id, task_id, comment_id, group_id, note_id, filename, mime_type, size, path, uploaded_by, created_at
FROM attachments WHERE group_id = ? ORDER BY created_at`, groupID)
if err != nil {
return nil, err
@@ -195,7 +210,7 @@ func (db *DB) attachmentsByComment(ctx context.Context, commentIDs []string) (ma
args[i] = id
}
rows, err := db.QueryContext(ctx,
`SELECT id, task_id, comment_id, group_id, filename, mime_type, size, path, uploaded_by, created_at
`SELECT id, task_id, comment_id, group_id, note_id, filename, mime_type, size, path, uploaded_by, created_at
FROM attachments WHERE comment_id IN (`+placeholders(len(commentIDs))+`)
ORDER BY created_at`, args...)
if err != nil {
@@ -218,15 +233,16 @@ func scanAttachments(rows *sql.Rows) ([]Attachment, error) {
out := []Attachment{}
for rows.Next() {
var a Attachment
var taskID, commentID, groupID sql.NullString
var taskID, commentID, groupID, noteID sql.NullString
var created int64
if err := rows.Scan(&a.ID, &taskID, &commentID, &groupID, &a.Filename, &a.MimeType,
if err := rows.Scan(&a.ID, &taskID, &commentID, &groupID, &noteID, &a.Filename, &a.MimeType,
&a.Size, &a.Path, &a.UploadedBy, &created); err != nil {
return nil, err
}
a.TaskID = taskID.String
a.CommentID = commentID.String
a.GroupID = groupID.String
a.NoteID = noteID.String
a.CreatedAt = time.Unix(created, 0).UTC()
out = append(out, a)
}
@@ -235,7 +251,7 @@ func scanAttachments(rows *sql.Rows) ([]Attachment, error) {
func (db *DB) GetAttachment(ctx context.Context, attachmentID string) (*Attachment, error) {
rows, err := db.QueryContext(ctx,
`SELECT id, task_id, comment_id, group_id, filename, mime_type, size, path, uploaded_by, created_at
`SELECT id, task_id, comment_id, group_id, note_id, filename, mime_type, size, path, uploaded_by, created_at
FROM attachments WHERE id = ?`, attachmentID)
if err != nil {
return nil, err
+253
View File
@@ -0,0 +1,253 @@
package store
import (
"context"
"database/sql"
"encoding/json"
"fmt"
"time"
)
// Mailbox is one connected mailbox belonging to one person.
type Mailbox struct {
ID string `json:"id"`
UserID string `json:"-"`
Kind string `json:"kind"` // "gmail" or "imap"
Name string `json:"name"`
Host string `json:"host,omitempty"`
Username string `json:"username,omitempty"`
Password string `json:"-"` // never leaves the server
Folder string `json:"folder,omitempty"`
RefreshToken string `json:"-"`
AccessToken string `json:"-"`
ExpiresAt time.Time `json:"-"`
Label string `json:"label,omitempty"`
// ProjectID is where this mailbox's tasks land. Empty means the inbox, and so
// does an id the person can no longer write to.
ProjectID string `json:"project_id,omitempty"`
Trigger string `json:"trigger,omitempty"` // "starred" or "label"
// Seen is the message ids Gmail has already turned into tasks. IMAP does not
// need it — uids are monotonic, so LastUID says the same in one number — but
// Gmail's query is a rolling window rather than a marker.
Seen []string `json:"-"`
LastUID uint32 `json:"last_uid,omitempty"`
LastSyncAt time.Time `json:"last_sync_at,omitempty"`
CreatedAt time.Time `json:"created_at"`
}
const mailboxColumns = `id, user_id, kind, name, host, username, password, folder,
refresh_token, access_token, expires_at, label, trigger_kind, seen, project_id,
last_uid, last_sync_at, created_at`
// Mailboxes returns everything one person has connected, oldest first.
func (db *DB) Mailboxes(ctx context.Context, userID string) ([]Mailbox, error) {
rows, err := db.QueryContext(ctx,
`SELECT `+mailboxColumns+` FROM mailboxes WHERE user_id = ? ORDER BY created_at`, userID)
if err != nil {
return nil, err
}
defer rows.Close()
var out []Mailbox
for rows.Next() {
m, err := db.scanMailbox(rows)
if err != nil {
return nil, err
}
out = append(out, m)
}
return out, rows.Err()
}
// Mailbox returns one, and only to the person it belongs to: the id is enough to
// find a row, so the owner is part of the question rather than checked after.
func (db *DB) Mailbox(ctx context.Context, userID, id string) (*Mailbox, error) {
row := db.QueryRowContext(ctx,
`SELECT `+mailboxColumns+` FROM mailboxes WHERE id = ? AND user_id = ?`, id, userID)
m, err := db.scanMailbox(row)
if err == sql.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, err
}
return &m, nil
}
// MailboxOfKind returns a person's mailbox of one kind, or nil. Gmail is one per
// person because the OAuth flow is: connecting again replaces what was there.
func (db *DB) MailboxOfKind(ctx context.Context, userID, kind string) (*Mailbox, error) {
row := db.QueryRowContext(ctx,
`SELECT `+mailboxColumns+` FROM mailboxes WHERE user_id = ? AND kind = ?`, userID, kind)
m, err := db.scanMailbox(row)
if err == sql.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, err
}
return &m, nil
}
// SaveMailbox writes one, sealing the two columns that must not travel with a
// backup. Called by hand rather than by the settings tables' own seal, which does
// not reach this table.
func (db *DB) SaveMailbox(ctx context.Context, m *Mailbox) error {
if m.ID == "" {
m.ID = NewID()
}
if m.CreatedAt.IsZero() {
m.CreatedAt = time.Now()
}
if m.Folder == "" {
m.Folder = "INBOX"
}
password, err := db.sealValue(m.Password)
if err != nil {
return fmt.Errorf("seal password: %w", err)
}
refresh, err := db.sealValue(m.RefreshToken)
if err != nil {
return fmt.Errorf("seal refresh_token: %w", err)
}
access, err := db.sealValue(m.AccessToken)
if err != nil {
return fmt.Errorf("seal access_token: %w", err)
}
// Not sealed: which mails have been read is not a secret, and a column of
// ciphertext here would only make the table harder to look at.
seen, err := json.Marshal(m.Seen)
if err != nil {
return err
}
if m.Trigger == "" {
m.Trigger = "starred"
}
_, err = db.ExecContext(ctx, `
INSERT INTO mailboxes (`+mailboxColumns+`)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (id) DO UPDATE SET
name = excluded.name,
host = excluded.host,
username = excluded.username,
password = excluded.password,
folder = excluded.folder,
refresh_token = excluded.refresh_token,
access_token = excluded.access_token,
expires_at = excluded.expires_at,
label = excluded.label,
trigger_kind = excluded.trigger_kind,
seen = excluded.seen,
project_id = excluded.project_id,
last_uid = excluded.last_uid,
last_sync_at = excluded.last_sync_at`,
m.ID, m.UserID, m.Kind, m.Name, m.Host, m.Username, password, m.Folder,
refresh, access, unixOrZero(m.ExpiresAt), m.Label, m.Trigger, string(seen), m.ProjectID,
m.LastUID, unixOrZero(m.LastSyncAt), m.CreatedAt.Unix())
return err
}
// MarkMailboxRead records how far a run got. Written separately from the rest so
// a sync never rewrites the credentials it is holding in memory.
func (db *DB) MarkMailboxRead(ctx context.Context, id string, lastUID uint32, at time.Time) error {
_, err := db.ExecContext(ctx,
`UPDATE mailboxes SET last_uid = ?, last_sync_at = ? WHERE id = ?`,
lastUID, at.Unix(), id)
return err
}
// DeleteMailbox disconnects one. The tasks it made stay: they are the person's
// work now, not the mailbox's.
func (db *DB) DeleteMailbox(ctx context.Context, userID, id string) error {
_, err := db.ExecContext(ctx, `DELETE FROM mailboxes WHERE id = ? AND user_id = ?`, id, userID)
return err
}
// UsersWithMailboxes returns everybody the sweep has to visit.
func (db *DB) UsersWithMailboxes(ctx context.Context) ([]string, error) {
rows, err := db.QueryContext(ctx, `SELECT DISTINCT user_id FROM mailboxes`)
if err != nil {
return nil, err
}
defer rows.Close()
var ids []string
for rows.Next() {
var id string
if err := rows.Scan(&id); err != nil {
return nil, err
}
ids = append(ids, id)
}
return ids, rows.Err()
}
type scanner interface {
Scan(dest ...any) error
}
func (db *DB) scanMailbox(row scanner) (Mailbox, error) {
var m Mailbox
var expires, lastSync, created int64
var seen string
err := row.Scan(&m.ID, &m.UserID, &m.Kind, &m.Name, &m.Host, &m.Username,
&m.Password, &m.Folder, &m.RefreshToken, &m.AccessToken, &expires,
&m.Label, &m.Trigger, &seen, &m.ProjectID, &m.LastUID, &lastSync, &created)
if err != nil {
return m, err
}
// A row that somehow holds nonsense here is not worth failing a whole sync
// over: an empty list means everything looks new, which is recoverable.
_ = json.Unmarshal([]byte(seen), &m.Seen)
for _, field := range []*string{&m.Password, &m.RefreshToken, &m.AccessToken} {
plain, err := db.unsealValue(*field)
if err != nil {
return m, err
}
*field = plain
}
m.ExpiresAt = timeOrZero(expires)
m.LastSyncAt = timeOrZero(lastSync)
m.CreatedAt = time.Unix(created, 0)
return m, nil
}
// sealValue and unsealValue are the single-string versions of the map helpers in
// notifications.go. Without a key they pass the value through, which is what the
// tests that never set one expect.
func (db *DB) sealValue(v string) (string, error) {
if db.box == nil {
return v, nil
}
return db.box.Seal(v)
}
func (db *DB) unsealValue(v string) (string, error) {
if db.box == nil {
return v, nil
}
return db.box.Unseal(v)
}
func unixOrZero(t time.Time) int64 {
if t.IsZero() {
return 0
}
return t.Unix()
}
func timeOrZero(unix int64) time.Time {
if unix == 0 {
return time.Time{}
}
return time.Unix(unix, 0)
}
+264
View File
@@ -0,0 +1,264 @@
package store
import (
"context"
"path/filepath"
"strings"
"testing"
"time"
"github.com/kristianwind/verdande/internal/secret"
)
func TestAMailboxIsStoredSealedAndComesBackWhole(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
const password = "abcd-efgh-ijkl-mnop"
m := &Mailbox{
UserID: userID,
Kind: "imap",
Name: "iCloud",
Host: "imap.mail.me.com:993",
Username: "[email protected]",
Password: password,
}
if err := db.SaveMailbox(ctx, m); err != nil {
t.Fatal(err)
}
var stored string
if err := db.QueryRowContext(ctx,
`SELECT password FROM mailboxes WHERE id = ?`, m.ID).Scan(&stored); err != nil {
t.Fatal(err)
}
if strings.Contains(stored, password) {
t.Fatalf("the password is in the row as typed: %s", stored)
}
back, err := db.Mailbox(ctx, userID, m.ID)
if err != nil {
t.Fatal(err)
}
if back == nil {
t.Fatal("the mailbox was not found")
}
if back.Password != password {
t.Errorf("got password %q", back.Password)
}
if back.Folder != "INBOX" {
t.Errorf("the folder defaulted to %q, want INBOX", back.Folder)
}
}
// The question that prompted the table: two people on one instance, each with
// their own mailbox, neither seeing the other's.
func TestAMailboxBelongsToOnePerson(t *testing.T) {
db, kristian := sealedStore(t)
ctx := context.Background()
andreas := &User{Email: "[email protected]", Name: "Andreas"}
if err := db.CreateUser(ctx, andreas, "Indbakke"); err != nil {
t.Fatal(err)
}
mine := &Mailbox{UserID: kristian, Kind: "imap", Host: "imap.mail.me.com:993", Username: "kw"}
theirs := &Mailbox{UserID: andreas.ID, Kind: "imap", Host: "imap.fastmail.com:993", Username: "andreas"}
for _, m := range []*Mailbox{mine, theirs} {
if err := db.SaveMailbox(ctx, m); err != nil {
t.Fatal(err)
}
}
list, err := db.Mailboxes(ctx, kristian)
if err != nil {
t.Fatal(err)
}
if len(list) != 1 || list[0].Username != "kw" {
t.Fatalf("got %d mailboxes for Kristian, want only his own", len(list))
}
// And the id alone is not enough to reach somebody else's.
other, err := db.Mailbox(ctx, kristian, theirs.ID)
if err != nil {
t.Fatal(err)
}
if other != nil {
t.Error("one person read another's mailbox by its id")
}
}
// Several mailboxes each is the ordinary case — a work Gmail and a private
// iCloud — and is the whole reason this is a table rather than a setting.
func TestOnePersonCanHaveSeveral(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
for _, host := range []string{"imap.mail.me.com:993", "imap.fastmail.com:993"} {
if err := db.SaveMailbox(ctx, &Mailbox{
UserID: userID, Kind: "imap", Host: host, Username: "kw",
}); err != nil {
t.Fatal(err)
}
}
list, err := db.Mailboxes(ctx, userID)
if err != nil {
t.Fatal(err)
}
if len(list) != 2 {
t.Fatalf("got %d mailboxes, want 2", len(list))
}
}
// The same mailbox twice would make two tasks out of every mail.
func TestTheSameMailboxTwiceIsRefused(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
first := &Mailbox{UserID: userID, Kind: "imap", Host: "imap.mail.me.com:993", Username: "kw"}
if err := db.SaveMailbox(ctx, first); err != nil {
t.Fatal(err)
}
again := &Mailbox{UserID: userID, Kind: "imap", Host: "imap.mail.me.com:993", Username: "kw"}
if err := db.SaveMailbox(ctx, again); err == nil {
t.Error("the same host and username was accepted a second time")
}
}
func TestReadingProgressIsRemembered(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
m := &Mailbox{UserID: userID, Kind: "imap", Host: "imap.mail.me.com:993", Username: "kw"}
if err := db.SaveMailbox(ctx, m); err != nil {
t.Fatal(err)
}
at := time.Now().Truncate(time.Second)
if err := db.MarkMailboxRead(ctx, m.ID, 4711, at); err != nil {
t.Fatal(err)
}
back, err := db.Mailbox(ctx, userID, m.ID)
if err != nil {
t.Fatal(err)
}
if back.LastUID != 4711 {
t.Errorf("last uid is %d", back.LastUID)
}
if !back.LastSyncAt.Equal(at) {
t.Errorf("last sync is %v, want %v", back.LastSyncAt, at)
}
// The credentials it was holding must not have been rewritten as a side effect.
if back.Host != "imap.mail.me.com:993" {
t.Errorf("the host changed to %q", back.Host)
}
}
// Disconnecting takes the mailbox, not the work it produced.
func TestDisconnectingKeepsTheTasks(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
m := &Mailbox{UserID: userID, Kind: "imap", Host: "imap.mail.me.com:993", Username: "kw"}
if err := db.SaveMailbox(ctx, m); err != nil {
t.Fatal(err)
}
if err := db.DeleteMailbox(ctx, userID, m.ID); err != nil {
t.Fatal(err)
}
list, err := db.Mailboxes(ctx, userID)
if err != nil {
t.Fatal(err)
}
if len(list) != 0 {
t.Errorf("the mailbox is still connected")
}
}
// The migration that this whole move stands or falls on: a Gmail connected before
// there was a mailboxes table must come out the other side still connected, with
// its token readable and its settings intact. Nobody should have to reconnect
// because the server tidied up.
func TestAGmailConnectedBeforeTheMoveSurvivesIt(t *testing.T) {
dir := t.TempDir()
box, err := secret.Open("", t.TempDir())
if err != nil {
t.Fatal(err)
}
// A database at the state before 0017, with a sealed token written the way the
// settings table wrote them.
db, userID := storeAt(t, dir, box)
sealed, err := box.Seal("1//0-the-original-refresh-token")
if err != nil {
t.Fatal(err)
}
values := `{"refresh_token":"` + sealed + `","trigger":"label","label":"Verdande",` +
`"email":"[email protected]","seen":["abc","def"],"project_id":"p-1"}`
if _, err := db.ExecContext(context.Background(), `
INSERT INTO user_settings (user_id, scope, values_json, updated_at)
VALUES (?, 'gmail', ?, 0)
ON CONFLICT (user_id) DO UPDATE SET scope = 'gmail', values_json = excluded.values_json`,
userID, values); err != nil {
t.Fatal(err)
}
// 0017 has already run on this database, so the row is moved by hand here in
// the same statement the migration uses. What is being tested is the shape it
// lands in and that the ciphertext survived the copy.
if _, err := db.ExecContext(context.Background(), `
INSERT INTO mailboxes (id, user_id, kind, name, host, username, password, folder,
refresh_token, access_token, expires_at, label, trigger_kind, seen, project_id,
last_uid, last_sync_at, created_at)
SELECT lower(hex(randomblob(16))), s.user_id, 'gmail',
json_extract(s.values_json, '$.email'), '', json_extract(s.values_json, '$.email'),
'', 'INBOX', json_extract(s.values_json, '$.refresh_token'), '', 0,
json_extract(s.values_json, '$.label'), json_extract(s.values_json, '$.trigger'),
json_extract(s.values_json, '$.seen'), json_extract(s.values_json, '$.project_id'),
0, 0, unixepoch()
FROM user_settings s WHERE s.scope = 'gmail'`); err != nil {
t.Fatal(err)
}
moved, err := db.MailboxOfKind(context.Background(), userID, "gmail")
if err != nil {
t.Fatal(err)
}
if moved == nil {
t.Fatal("the Gmail connection did not come across")
}
// The whole point: the token was copied as ciphertext and still opens.
if moved.RefreshToken != "1//0-the-original-refresh-token" {
t.Errorf("the token came out as %q", moved.RefreshToken)
}
if moved.Trigger != "label" || moved.Label != "Verdande" {
t.Errorf("the trigger came out as %q/%q", moved.Trigger, moved.Label)
}
if moved.Username != "[email protected]" {
t.Errorf("the address came out as %q", moved.Username)
}
if moved.ProjectID != "p-1" {
t.Errorf("the destination project came out as %q", moved.ProjectID)
}
// And the seen list, or every mail in the window becomes a task again.
if len(moved.Seen) != 2 || moved.Seen[0] != "abc" {
t.Errorf("the seen list came out as %v", moved.Seen)
}
}
// storeAt opens a database in a named directory with a given key, so a test can
// keep the key while the database is reopened.
func storeAt(t *testing.T, dir string, box *secret.Box) (*DB, string) {
t.Helper()
db, err := Open(filepath.Join(dir, "test.db"))
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { db.Close() })
db.UseSecrets(box)
user := &User{Email: "[email protected]", Name: "Kristian"}
if err := db.CreateUser(context.Background(), user, "Indbakke"); err != nil {
t.Fatal(err)
}
return db, user.ID
}
@@ -0,0 +1,57 @@
-- Mailboxes: one row per connected mailbox, not one setting per person.
--
-- Gmail lived in `user_settings` under scope 'gmail'. That table has `user_id` as
-- its whole primary key, so it holds exactly one scope per person and its upsert
-- rewrites the scope name on conflict — adding IMAP beside Gmail there would have
-- deleted the Gmail connection without saying so. Migration 0011 warns about this
-- in as many words, and the warning was nearly walked into a second time.
--
-- The shape is wrong for the thing anyway. A person has mailboxes, plural: a work
-- Gmail and a private iCloud is the ordinary case, not the exotic one. A row each
-- says that. It also answers the question that prompted this — every mailbox
-- belongs to a user, so two people on the same instance connect their own and see
-- only their own. The one piece that stays instance-wide is the Google OAuth
-- client, which is a single registration everybody signs in through, and that is
-- correct: it is the app, not the account.
--
-- `kind` rather than two tables. What differs is how the credential is obtained —
-- OAuth for Gmail, a password for IMAP — and what differs is a nullable column,
-- not a schema. What is the same is everything the sync loop touches: which
-- mailbox, how far it has read, when it last ran.
--
-- `password` and `refresh_token` are written sealed; see internal/secret. The
-- store's own seal/unseal only covers the settings tables, so this table's methods
-- call them by hand — a fact worth knowing before adding a third secret column.
--
-- `last_uid` is IMAP's place-marker and means nothing for Gmail, which keeps its
-- own by query. Zero is "nothing read yet" for both.
CREATE TABLE mailboxes (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users (id) ON DELETE CASCADE,
kind TEXT NOT NULL CHECK (kind IN ('gmail', 'imap')),
name TEXT NOT NULL DEFAULT '',
-- IMAP
host TEXT NOT NULL DEFAULT '',
username TEXT NOT NULL DEFAULT '',
password TEXT NOT NULL DEFAULT '',
folder TEXT NOT NULL DEFAULT 'INBOX',
-- Gmail
refresh_token TEXT NOT NULL DEFAULT '',
access_token TEXT NOT NULL DEFAULT '',
expires_at INTEGER NOT NULL DEFAULT 0,
label TEXT NOT NULL DEFAULT '',
last_uid INTEGER NOT NULL DEFAULT 0,
last_sync_at INTEGER NOT NULL DEFAULT 0,
created_at INTEGER NOT NULL
);
-- The sweep asks "whose mailboxes are due", so it reads by user.
CREATE INDEX mailboxes_by_user ON mailboxes (user_id);
-- The same mailbox twice is a mistake, not a choice: two rows would make two
-- tasks out of every mail. Gmail's uniqueness is the account, IMAP's is the host
-- and the username together, and `kind` keeps the two from colliding.
CREATE UNIQUE INDEX mailboxes_once ON mailboxes (user_id, kind, host, username);
@@ -0,0 +1,22 @@
-- Room for Gmail beside the others.
--
-- Only the two columns. Moving the data is a separate migration on purpose: the
-- Gmail handlers still read user_settings, and a migration that emptied that table
-- before they were rewritten would disconnect a working mailbox. Additive first,
-- and the move when the code on the other side is ready for it.
--
-- Gmail lived in `user_settings` under scope 'gmail' because it was the only
-- integration of its kind. Now there are two, and two places to look for the same
-- question — which mailboxes does this person have — is one place too many.
--
-- `seen` is Gmail's own deduplication: a list of message ids it has already turned
-- into tasks. IMAP does not need it, because uids are monotonic and `last_uid`
-- answers the same question in one number. Gmail's query is a 30-day window rather
-- than a marker, so the list is what it has.
ALTER TABLE mailboxes ADD COLUMN seen TEXT NOT NULL DEFAULT '[]';
-- What makes a mail into a task: 'starred', or 'label' with the name in `label`.
-- Its own column rather than packed into `label` with a separator — a column that
-- holds two things joined by a colon is a column somebody parses wrongly later,
-- and a label named with a colon in it would do it on the first day.
ALTER TABLE mailboxes ADD COLUMN trigger_kind TEXT NOT NULL DEFAULT 'starred';
@@ -0,0 +1,12 @@
-- Where a mailbox's tasks land.
--
-- Gmail carried this as `project_id` in its settings blob. It belongs to the
-- mailbox, and it is worth having for the ones read over IMAP too: a work mailbox
-- filing into the work project is the obvious thing to want, and the alternative
-- is everything landing in the inbox to be moved by hand.
--
-- Empty means the person's inbox, which is also what an id that no longer names a
-- project they can write to falls back to. Not a foreign key for that reason: a
-- deleted project should quietly stop being the destination, not refuse the delete
-- or take the mailbox down with it.
ALTER TABLE mailboxes ADD COLUMN project_id TEXT NOT NULL DEFAULT '';
@@ -0,0 +1,46 @@
-- Gmail's rows move in beside the ones read over IMAP.
--
-- Held back until now on purpose: this empties the settings row, and running it
-- while the handlers still read that row would have disconnected a working
-- mailbox. internal/httpapi now reads the mailboxes table, so the move is safe.
--
-- The values move across still sealed. Both tables seal with the same key and
-- store the same string, so copying the ciphertext verbatim is not a shortcut
-- around the encryption — it is the encryption, untouched. That also means this
-- migration does not need the key, which a migration has no business holding.
--
-- `username` is the address the account reports, and it is what makes the unique
-- index meaningful. It may be empty on rows that never got that far; two such rows
-- cannot exist for one person anyway, because the settings table could only hold
-- one.
INSERT INTO mailboxes (
id, user_id, kind, name, host, username, password, folder,
refresh_token, access_token, expires_at, label, trigger_kind, seen, project_id,
last_uid, last_sync_at, created_at
)
SELECT
lower(hex(randomblob(16))),
s.user_id,
'gmail',
COALESCE(json_extract(s.values_json, '$.email'), 'Gmail'),
'',
COALESCE(json_extract(s.values_json, '$.email'), ''),
'',
'INBOX',
COALESCE(json_extract(s.values_json, '$.refresh_token'), ''),
COALESCE(json_extract(s.values_json, '$.access_token'), ''),
COALESCE(json_extract(s.values_json, '$.expires_at'), 0),
COALESCE(json_extract(s.values_json, '$.label'), ''),
COALESCE(json_extract(s.values_json, '$.trigger'), 'starred'),
COALESCE(json_extract(s.values_json, '$.seen'), '[]'),
COALESCE(json_extract(s.values_json, '$.project_id'), ''),
0,
0,
unixepoch()
FROM user_settings s
WHERE s.scope = 'gmail'
AND COALESCE(json_extract(s.values_json, '$.refresh_token'), '') <> '';
-- Emptied, not deleted: the row may hold nothing else today, but the table is
-- shared and a DELETE here would be a habit worth not forming.
UPDATE user_settings SET values_json = '{}' WHERE scope = 'gmail';
+88
View File
@@ -0,0 +1,88 @@
-- Notes.
--
-- Markdown in a column, not rich text and not a format of our own. Export was a
-- requirement before a line was written, and if the stored form *is* the exported
-- form there is no conversion left to lose anything in — a note stays readable by
-- any editor on the day this program is gone. It is also what makes full-text
-- search and anything an AI does over notes cheap: both read text.
--
-- Under a project rather than in a folder tree of its own. Projects, groups and
-- labels already exist and are already shared, already have roles, already have a
-- trash. A second hierarchy to file things in is one too many, and a note that
-- belongs to a project inherits every one of those answers instead of needing its
-- own. A note with no project is loose, the way a task in the inbox is.
--
-- `deleted_at` and `created_by` follow tasks exactly, including SET NULL: an
-- account can be removed without taking somebody else's notes with it.
CREATE TABLE notes (
id TEXT PRIMARY KEY,
project_id TEXT REFERENCES projects (id) ON DELETE CASCADE,
title TEXT NOT NULL DEFAULT '',
body TEXT NOT NULL DEFAULT '',
created_by TEXT REFERENCES users (id) ON DELETE SET NULL,
pinned INTEGER NOT NULL DEFAULT 0,
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL,
deleted_at INTEGER,
-- The same Danish transliteration as tasks carry, and for the same reason:
-- unicode61 folds diacritics, so "grøn" is findable as "gron", but it does
-- nothing for æ→ae or å→aa, which is how Danes actually type when the keyboard
-- is not theirs.
fold TEXT GENERATED ALWAYS AS (
replace(replace(replace(replace(replace(replace(
title || ' ' || body,
'ø', 'o'), 'Ø', 'o'),
'æ', 'ae'), 'Æ', 'ae'),
'å', 'aa'), 'Å', 'aa')
) STORED
);
CREATE INDEX notes_by_project ON notes (project_id, deleted_at, updated_at DESC);
-- What a note points at: a task, another note, a project.
--
-- Derived from the body every time it is saved, never edited directly. The text
-- stays the truth; this is an index over it — which is what makes the question
-- worth asking backwards. "What refers to this task" must not mean searching every
-- note's prose, or the panel that shows it would be too slow to leave open.
--
-- `target_id` is deliberately not a foreign key. A link to something that has been
-- deleted is a fact about the note, and it should read as a dead link rather than
-- vanish or block the delete.
CREATE TABLE note_links (
note_id TEXT NOT NULL REFERENCES notes (id) ON DELETE CASCADE,
kind TEXT NOT NULL CHECK (kind IN ('task', 'note', 'project')),
target_id TEXT NOT NULL,
PRIMARY KEY (note_id, kind, target_id)
);
CREATE INDEX note_links_backwards ON note_links (kind, target_id);
CREATE VIRTUAL TABLE notes_fts USING fts5 (
title,
body,
fold,
id UNINDEXED,
project_id UNINDEXED,
content = 'notes',
content_rowid = 'rowid',
tokenize = 'unicode61 remove_diacritics 2'
);
CREATE TRIGGER notes_fts_insert AFTER INSERT ON notes BEGIN
INSERT INTO notes_fts (rowid, title, body, fold, id, project_id)
VALUES (new.rowid, new.title, new.body, new.fold, new.id, new.project_id);
END;
CREATE TRIGGER notes_fts_delete AFTER DELETE ON notes BEGIN
INSERT INTO notes_fts (notes_fts, rowid, title, body, fold, id, project_id)
VALUES ('delete', old.rowid, old.title, old.body, old.fold, old.id, old.project_id);
END;
CREATE TRIGGER notes_fts_update AFTER UPDATE ON notes BEGIN
INSERT INTO notes_fts (notes_fts, rowid, title, body, fold, id, project_id)
VALUES ('delete', old.rowid, old.title, old.body, old.fold, old.id, old.project_id);
INSERT INTO notes_fts (rowid, title, body, fold, id, project_id)
VALUES (new.rowid, new.title, new.body, new.fold, new.id, new.project_id);
END;
@@ -0,0 +1,13 @@
-- The order of the fixed views in the sidebar.
--
-- On the account rather than in the browser, and for the reason the sidebar's
-- width is the other way round: a width is a fact about the screen you are at, and
-- the answer on a phone is not the answer on a monitor. Which order you want I dag,
-- Kommende, Venter and Indbakken in is a fact about you, and it should be the same
-- on the laptop as on the phone.
--
-- A list of keys rather than a number per row, for the same reason sidebar_collapsed
-- is: the set of views changes with the program, and a key that no longer exists
-- costs nothing — the sidebar draws what it recognises and appends anything new at
-- the end. That is what makes adding a view later not a migration.
ALTER TABLE users ADD COLUMN nav_order TEXT NOT NULL DEFAULT '[]';
@@ -0,0 +1,29 @@
-- #garageristeriet and #GarageRisteriet are the same project.
--
-- They were not, because the key was stored exactly as typed and looked up the
-- same way — so a note that named a project in lower case simply did not appear
-- on it, silently and with nothing to suggest why.
--
-- Folded on the way in from now on; this brings the rows already written into
-- line. Only projects: a [[note title]] is a title, and two notes called "Møde"
-- and "møde" are two notes.
--
-- The primary key is (note_id, kind, target_id), so folding can collide with a row
-- that is already there — a note that wrote both spellings. INSERT OR IGNORE into
-- a fresh set and swap, which keeps one of each rather than failing on the pair.
CREATE TABLE note_links_folded (
note_id TEXT NOT NULL REFERENCES notes (id) ON DELETE CASCADE,
kind TEXT NOT NULL CHECK (kind IN ('task', 'note', 'project')),
target_id TEXT NOT NULL,
PRIMARY KEY (note_id, kind, target_id)
);
INSERT OR IGNORE INTO note_links_folded (note_id, kind, target_id)
SELECT note_id, kind,
CASE WHEN kind = 'project' THEN lower(target_id) ELSE target_id END
FROM note_links;
DROP TABLE note_links;
ALTER TABLE note_links_folded RENAME TO note_links;
CREATE INDEX note_links_backwards ON note_links (kind, target_id);
@@ -0,0 +1,45 @@
-- A note can carry files, the way a task and a group already can.
--
-- Apple Notes is full of photographs, scans and pasted screenshots, and a note
-- that arrives here as words alone is not that note — it is a summary of it. The
-- import writes the files as attachments and rewrites the links in the text, so a
-- picture that was in the middle of a paragraph is still in the middle of it.
--
-- Same table as everything else attaches to, rather than one of its own. The files
-- are content-addressed and deduplicated on disk, the upload path is written and
-- tested, and the cleanup that removes an orphan already walks this table. A
-- second table would be a second everything.
-- verdande:rebuild-tables
CREATE TABLE attachments_with_notes (
id TEXT PRIMARY KEY,
task_id TEXT REFERENCES tasks (id) ON DELETE CASCADE,
comment_id TEXT REFERENCES comments (id) ON DELETE CASCADE,
group_id TEXT REFERENCES project_groups (id) ON DELETE CASCADE,
note_id TEXT REFERENCES notes (id) ON DELETE CASCADE,
filename TEXT NOT NULL,
mime_type TEXT NOT NULL DEFAULT 'application/octet-stream',
size INTEGER NOT NULL,
path TEXT NOT NULL,
uploaded_by TEXT NOT NULL REFERENCES users (id) ON DELETE CASCADE,
created_at INTEGER NOT NULL,
-- Counted, not spelled out as pairs: with four columns the pairwise version is
-- six clauses, and the one that is wrong looks exactly like the one that is right.
CHECK ((task_id IS NOT NULL) + (comment_id IS NOT NULL)
+ (group_id IS NOT NULL) + (note_id IS NOT NULL) = 1)
);
INSERT INTO attachments_with_notes
(id, task_id, comment_id, group_id, filename, mime_type, size, path, uploaded_by, created_at)
SELECT id, task_id, comment_id, group_id, filename, mime_type, size, path, uploaded_by, created_at
FROM attachments;
DROP TABLE attachments;
ALTER TABLE attachments_with_notes RENAME TO attachments;
CREATE INDEX idx_attachments_task ON attachments (task_id) WHERE task_id IS NOT NULL;
CREATE INDEX idx_attachments_comment ON attachments (comment_id) WHERE comment_id IS NOT NULL;
CREATE INDEX idx_attachments_group ON attachments (group_id) WHERE group_id IS NOT NULL;
CREATE INDEX idx_attachments_note ON attachments (note_id) WHERE note_id IS NOT NULL;
+26
View File
@@ -0,0 +1,26 @@
-- One row per installation that has ever reported in.
--
-- This table only exists on the instance that is acting as the collector. Every
-- other instance has it too, empty, because the schema is the same everywhere —
-- and because "am I the collector" is a setting, not a build.
--
-- What is deliberately NOT here is the point of the whole feature: there is no
-- column for an IP address, a hostname, a domain, an account, a count of tasks,
-- or anything else. The two things stored are the two things sent, and the table
-- is the promise written down where it cannot drift from the prose — a column
-- added here later is a promise broken, and it should be as awkward to add as
-- possible.
--
-- `instance_id` is the primary key rather than a rowid alias, because the same
-- installation reporting daily must update its row rather than grow a new one.
-- What is wanted is "how many installations", not "how many pings".
CREATE TABLE beacon_installs (
instance_id TEXT PRIMARY KEY,
version TEXT NOT NULL DEFAULT '',
first_seen INTEGER NOT NULL,
last_seen INTEGER NOT NULL
);
-- The count is always "seen in the last N days", so last_seen is what every query
-- sorts and filters on.
CREATE INDEX beacon_by_last_seen ON beacon_installs (last_seen DESC);
@@ -0,0 +1,111 @@
-- Calendars read from somewhere else: the account, the calendars in it, and the
-- events they hold.
--
-- Not in `mailboxes`, and it was the first thing considered. The two look alike
-- from a distance — a person connects a Google account over OAuth, tokens are
-- sealed, a sweep polls it — and every column past that disagrees. `trigger_kind`,
-- `seen`, `last_uid` and `project_id` all answer "what turns a message into a
-- task", and nothing here turns into a task. What is here instead is a *set*:
-- one account holds many calendars, and a person picks which of them to look at.
-- `mailboxes` has no room for a one-to-many, and giving it one would push the
-- other kind's columns onto rows that can never use them.
--
-- The decisive part is smaller and more practical. `mailboxes.kind` carries
-- `CHECK (kind IN ('gmail', 'imap'))`, and SQLite cannot alter a CHECK — adding a
-- third value means rebuilding the table with foreign keys off, which is the
-- expensive, careful thing migration 0021 had to do for a reason. Paying that to
-- store something that shares no column with the table is the wrong trade.
--
-- Hence no CHECK on `provider` here. CalDAV is the next source to read, and it
-- should cost a row, not a rebuild.
-- One connected account per person per provider. The tokens live here rather than
-- on each calendar: a refresh token copied onto ten rows is ten copies of one
-- secret, ten refreshes racing each other, and ten places to miss when access is
-- revoked.
--
-- `refresh_token` and `access_token` are written sealed; see internal/secret. The
-- store's own seal/unseal only covers the settings tables, so this table's methods
-- call them by hand — the same fact worth knowing that migration 0014 wrote down.
CREATE TABLE calendar_accounts (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users (id) ON DELETE CASCADE,
provider TEXT NOT NULL DEFAULT 'google',
-- The address the account reports, so the settings page can say which one is
-- connected rather than only that one is.
account TEXT NOT NULL DEFAULT '',
refresh_token TEXT NOT NULL DEFAULT '',
access_token TEXT NOT NULL DEFAULT '',
expires_at INTEGER NOT NULL DEFAULT 0,
last_sync_at INTEGER NOT NULL DEFAULT 0,
created_at INTEGER NOT NULL
);
-- Connecting again replaces what was there, the way Gmail's does: the OAuth flow
-- is one account at a time, so a second row for the same provider would be a
-- second set of tokens for the same grant and no way to say which is live.
CREATE UNIQUE INDEX calendar_accounts_once ON calendar_accounts (user_id, provider);
-- One row per calendar the account can see, kept whether or not it is shown.
--
-- `shown` rather than deleting the ones that are not: a person who turns a calendar
-- off and on again should get the same row back, and the list is refreshed from
-- Google on every sync — so a calendar that only existed while it was ticked would
-- lose its choice every time somebody added a new one.
--
-- `colour` is Google's own hex for the calendar. Not one of verdande's ten named
-- colours, which is the rule everywhere else in this database: those are names so
-- the interface can restyle them per theme, and they are *verdande's* palette. A
-- Google calendar is told apart by the colour the person already knows it by, and
-- that colour belongs to Google.
CREATE TABLE calendars (
id TEXT PRIMARY KEY,
account_id TEXT NOT NULL REFERENCES calendar_accounts (id) ON DELETE CASCADE,
-- The id Google knows it by. Long, and an address for the primary calendar.
remote_id TEXT NOT NULL,
name TEXT NOT NULL DEFAULT '',
colour TEXT NOT NULL DEFAULT '',
time_zone TEXT NOT NULL DEFAULT '',
primary_one INTEGER NOT NULL DEFAULT 0,
writable INTEGER NOT NULL DEFAULT 0,
shown INTEGER NOT NULL DEFAULT 0,
sort_order INTEGER NOT NULL DEFAULT 0
);
CREATE UNIQUE INDEX calendars_once ON calendars (account_id, remote_id);
-- The events themselves, cached.
--
-- Cached rather than fetched per view, and the reason is what a month grid does:
-- it pages. Asking Google on every arrow press means a token refresh, a round trip
-- and a spinner for a question that was answered ninety seconds ago — and a
-- calendar that is slower than the calendar it is showing is not worth having.
-- The window is refreshed by the sweep and by the "fetch now" button.
--
-- Days rather than instants, and both ends inclusive. A grid asks one question —
-- "which cells does this cover" — and `start_day <= ? AND end_day >= ?` is that
-- question. `starts_at` is kept as Google wrote it, offset and all, because it is
-- what the chip shows and reparsing it here would ask *this process* what time
-- zone it is in. An all-day event has neither: it has a day, not a moment, and
-- inventing midnight for it is how an event moves a day when a container moves
-- time zone.
CREATE TABLE calendar_events (
id TEXT PRIMARY KEY,
calendar_id TEXT NOT NULL REFERENCES calendars (id) ON DELETE CASCADE,
remote_id TEXT NOT NULL,
summary TEXT NOT NULL DEFAULT '',
starts_at TEXT NOT NULL DEFAULT '',
ends_at TEXT NOT NULL DEFAULT '',
start_day TEXT NOT NULL,
end_day TEXT NOT NULL,
all_day INTEGER NOT NULL DEFAULT 0,
location TEXT NOT NULL DEFAULT '',
url TEXT NOT NULL DEFAULT ''
);
-- The one query there is: everything a person can see, between two days. Ordered
-- by the account so the join has somewhere to start, then by the days the grid
-- compares against.
CREATE INDEX calendar_events_by_day ON calendar_events (calendar_id, start_day, end_day);
@@ -0,0 +1,23 @@
-- A note can be put away without being thrown away.
--
-- The trash already exists and is the wrong tool. It says "this was a mistake,
-- and in thirty days it is gone" — so anything worth keeping cannot go in it, and
-- an archive of twelve hundred imported notes is exactly the case where most of
-- them are worth keeping and almost none are worth looking at. Without somewhere
-- to put them, the list is the whole archive forever, and finding this week's note
-- means reading past a decade.
--
-- A column rather than a folder. Verdande already refused a second hierarchy for
-- notes — see 0018 — and archiving is not filing: it is one bit about whether a
-- note is current, and it has to be answerable next to `deleted_at`, which is the
-- other bit of the same kind.
--
-- Nullable timestamp rather than a boolean, for the same reason `deleted_at` is
-- one: "when was this put away" answers questions a flag cannot, and it costs the
-- same byte.
ALTER TABLE notes ADD COLUMN archived_at INTEGER;
-- The list asks "not deleted, not archived, newest first" on every load, and with
-- twelve hundred notes that is the query that has to be quick.
CREATE INDEX notes_current ON notes (archived_at, deleted_at, updated_at DESC)
WHERE deleted_at IS NULL;
+434
View File
@@ -0,0 +1,434 @@
package store
import (
"context"
"database/sql"
"regexp"
"strings"
"time"
)
// Note is a page of Markdown belonging to a project, or to nobody.
type Note struct {
ID string `json:"id"`
ProjectID string `json:"project_id,omitempty"`
Title string `json:"title"`
Body string `json:"body"`
CreatedBy string `json:"created_by,omitempty"`
Pinned bool `json:"pinned"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
DeletedAt *time.Time `json:"deleted_at,omitempty"`
ArchivedAt *time.Time `json:"archived_at,omitempty"`
// Preview is the first of the body, for a list that shows one line of it.
//
// It exists so a list of twelve hundred notes does not have to carry twelve
// hundred whole notes: the list rendered 845 KB of JSON to show ninety
// characters of each. When it is set, Body is deliberately empty — see
// handleListNotes. That is not tidiness but a guard: an editor handed a
// truncated body would save the truncation, and the note would be cut down to
// its own preview.
Preview string `json:"preview,omitempty"`
// Links is what the body points at, read out of the text on the way in. Never
// set by a caller: it is derived, and a caller that set it would be describing
// a note whose text says something else.
Links []NoteLink `json:"links,omitempty"`
}
// NoteLink is one reference from a note to something else.
type NoteLink struct {
Kind string `json:"kind"` // "task", "note" or "project"
TargetID string `json:"target_id"`
}
const noteColumns = `id, project_id, title, body, created_by, pinned,
created_at, updated_at, deleted_at, archived_at`
// The same columns, said with the table in front of them.
//
// Needed only by the search, which joins `notes_fts` — and that table carries its
// own `id` and `project_id` (UNINDEXED, so the row can be found from a match).
// Unqualified, SQLite cannot tell which one is meant and refuses the query.
const noteColumnsQualified = `notes.id, notes.project_id, notes.title, notes.body,
notes.created_by, notes.pinned, notes.created_at, notes.updated_at, notes.deleted_at,
notes.archived_at`
// Reference syntax, and deliberately the same characters the quick-add parser
// already uses. A `#Firma` in a note means the same project as a `#Firma` typed
// into the task box — not a second kind of tag that happens to look alike. That
// is the whole of "tag a note and see it in the project".
var (
projectRef = regexp.MustCompile(`(?:^|\s)#([\p{L}\p{N}_-]{1,64})`)
// [[a note]] by title, the spelling every note-taking program has settled on.
noteRef = regexp.MustCompile(`\[\[([^\]\n]{1,200})\]\]`)
// A task by id, which is what a link pasted out of the app carries.
taskRef = regexp.MustCompile(`(?:/opgave/|task:)([0-9a-fA-F-]{36})`)
)
// SaveNote writes a note and rebuilds what it points at.
//
// The links are torn down and rebuilt rather than diffed. A note's body is small,
// the set is small, and a diff would be a second description of the same fact —
// one that can disagree with the text, which is the one thing this must never do.
func (db *DB) SaveNote(ctx context.Context, n *Note) error {
if n.ID == "" {
n.ID = NewID()
}
now := time.Now()
if n.CreatedAt.IsZero() {
n.CreatedAt = now
}
n.UpdatedAt = now
// The title is always the first line, the way Apple Notes does it. Not "when it
// is empty": derived once and then left alone, it goes stale the moment somebody
// rewrites the opening of a note, and the list ends up calling a note by a name
// that is nowhere in it. The column is a cache of the body, not a field beside
// it, and every caller gets the same answer because none of them can set it.
n.Title = firstLine(n.Body)
return db.Tx(ctx, func(tx *sql.Tx) error {
var projectID any
if n.ProjectID != "" {
projectID = n.ProjectID
}
var createdBy any
if n.CreatedBy != "" {
createdBy = n.CreatedBy
}
if _, err := tx.ExecContext(ctx, `
INSERT INTO notes (id, project_id, title, body, created_by, pinned, created_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (id) DO UPDATE SET
project_id = excluded.project_id,
title = excluded.title,
body = excluded.body,
pinned = excluded.pinned,
updated_at = excluded.updated_at`,
n.ID, projectID, n.Title, n.Body, createdBy, n.Pinned,
n.CreatedAt.Unix(), n.UpdatedAt.Unix()); err != nil {
return err
}
if _, err := tx.ExecContext(ctx, `DELETE FROM note_links WHERE note_id = ?`, n.ID); err != nil {
return err
}
n.Links = LinksIn(n.Body)
for _, l := range n.Links {
if _, err := tx.ExecContext(ctx,
`INSERT OR IGNORE INTO note_links (note_id, kind, target_id) VALUES (?, ?, ?)`,
n.ID, l.Kind, l.TargetID); err != nil {
return err
}
}
return nil
})
}
// LinksIn reads the references out of a body.
//
// Exported because it is worth testing on its own, and because the editor wants
// the same answer while somebody is typing rather than after they have saved.
func LinksIn(body string) []NoteLink {
seen := map[NoteLink]bool{}
var out []NoteLink
add := func(kind, target string) {
l := NoteLink{Kind: kind, TargetID: target}
if target == "" || seen[l] {
return
}
seen[l] = true
out = append(out, l)
}
for _, m := range taskRef.FindAllStringSubmatch(body, -1) {
add("task", strings.ToLower(m[1]))
}
for _, m := range projectRef.FindAllStringSubmatch(body, -1) {
// Folded, because #garageristeriet and #GarageRisteriet are the same project
// to everybody except a database. The key is stored folded and looked up
// folded, so the comparison stays an exact match and keeps using its index —
// a comparison that lowered the column instead would read the whole table.
//
// The project's real spelling is not lost: it is on the project, which is
// what the interface shows. This column is a key, not a label.
add("project", strings.ToLower(m[1]))
}
for _, m := range noteRef.FindAllStringSubmatch(body, -1) {
add("note", strings.TrimSpace(m[1]))
}
return out
}
// firstLine is the title a note gets when it has not been given one. Markdown
// heading marks are stripped: "# Møde" is titled "Møde", not "# Møde".
func firstLine(body string) string {
for _, line := range strings.Split(body, "\n") {
line = strings.TrimSpace(strings.TrimLeft(strings.TrimSpace(line), "#"))
if line != "" {
if len([]rune(line)) > 120 {
return string([]rune(line)[:120])
}
return line
}
}
return ""
}
// Note returns one, or nil when it is not there or is in the trash.
func (db *DB) Note(ctx context.Context, id string) (*Note, error) {
row := db.QueryRowContext(ctx,
`SELECT `+noteColumns+` FROM notes WHERE id = ? AND deleted_at IS NULL`, id)
n, err := scanNote(row)
if err == sql.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, err
}
links, err := db.linksOf(ctx, n.ID)
if err != nil {
return nil, err
}
n.Links = links
return &n, nil
}
// NotesInProject lists a project's notes, pinned first and newest after.
func (db *DB) NotesInProject(ctx context.Context, projectID string) ([]Note, error) {
return db.notesWhere(ctx,
`WHERE project_id = ? AND deleted_at IS NULL ORDER BY pinned DESC, updated_at DESC`,
projectID)
}
// LooseNotes are the ones filed under no project — a person's own, the way a task
// in the inbox is. Nobody else can see them, so there is no role to ask about.
func (db *DB) LooseNotes(ctx context.Context, userID string) ([]Note, error) {
return db.notesWhere(ctx, `
WHERE project_id IS NULL AND created_by = ? AND deleted_at IS NULL
ORDER BY pinned DESC, updated_at DESC`, userID)
}
// AllNotes is everything a person can see: their own loose ones, and the ones in
// projects they are part of.
//
// This is what the Notes page lists. Filing a note in a project is how it gets
// shared, and a list that dropped it at that moment would make sharing look like
// deleting — which is exactly how it read before this existed.
func (db *DB) AllNotes(ctx context.Context, userID string) ([]Note, error) {
return db.allNotes(ctx, userID, false)
}
// ArchivedNotes is the same list, the other way round: what has been put away.
func (db *DB) ArchivedNotes(ctx context.Context, userID string) ([]Note, error) {
return db.allNotes(ctx, userID, true)
}
func (db *DB) allNotes(ctx context.Context, userID string, archived bool) ([]Note, error) {
// Archived notes are left out of the ordinary list rather than mixed in and
// greyed: the point of putting something away is not seeing it.
state := "notes.archived_at IS NULL"
if archived {
state = "notes.archived_at IS NOT NULL"
}
return db.notesJoin(ctx, noteColumnsQualified, `
WHERE `+state+` AND deleted_at IS NULL
AND (
(project_id IS NULL AND created_by = ?)
OR project_id IN (
SELECT id FROM projects WHERE owner_id = ? AND deleted_at IS NULL
UNION
SELECT project_id FROM project_members WHERE user_id = ?
)
)
ORDER BY pinned DESC, updated_at DESC`, userID, userID, userID)
}
// NotesLinking answers the backwards question: what points at this thing.
//
// The reason note_links exists. Without it this would mean reading every note's
// prose, and a panel that slow is one nobody leaves open.
func (db *DB) NotesLinking(ctx context.Context, kind, targetID string) ([]Note, error) {
if kind == "project" {
targetID = strings.ToLower(targetID)
}
return db.notesWhere(ctx, `
WHERE deleted_at IS NULL
AND id IN (SELECT note_id FROM note_links WHERE kind = ? AND target_id = ?)
ORDER BY updated_at DESC`, kind, targetID)
}
// SearchNotes matches title and body, in either spelling of a Danish word.
func (db *DB) SearchNotes(ctx context.Context, query string, limit int) ([]Note, error) {
if strings.TrimSpace(query) == "" {
return nil, nil
}
if limit <= 0 {
limit = 50
}
expr := MatchExprOver(query, "title", "body")
if expr == "" {
return nil, nil
}
// Ordered by how well it matched, not by when it was last touched.
//
// `ORDER BY updated_at DESC` was invisible while there were ten notes and
// useless at twelve hundred: every one of them was imported the same evening,
// so `updated_at` says nothing, and the limit then cut the list at fifty
// arbitrary rows. Searching "Tesla" answered with a note about nginx first and
// the note actually called "Tesla Model Y" second — which reads as a search
// that does not work, because from where the person is sitting it is one.
//
// bm25 with the title weighted ten times the body. A word in the title is what
// the note is *about*; the same word once in a long body is a mention. `fold`
// carries the same text transliterated, so it must weigh the same as the body
// or a Danish match would outrank an exact one.
//
// Joined rather than `rowid IN (…)`: the rank belongs to the match, and a
// subquery throws it away.
return db.notesJoin(ctx, noteColumnsQualified, `
JOIN notes_fts ON notes_fts.rowid = notes.rowid
WHERE notes.deleted_at IS NULL
AND notes_fts MATCH ?
ORDER BY bm25(notes_fts, 10.0, 1.0, 1.0)
LIMIT ?`, expr, limit)
}
// SetNoteTimes writes the dates a note had somewhere else.
//
// Only the import calls this, and it exists because SaveNote is right to refuse:
// `updated_at` is when this program last wrote the note, and letting any caller
// set it would make the column mean whatever the last writer felt like. A note
// brought in from Apple Notes is the one case where the dates are facts from
// before this database existed — and without them a move of twelve hundred notes
// is a pile that was all written the same evening, which throws away the order
// they were written in. That order is half of what an archive is.
func (db *DB) SetNoteTimes(ctx context.Context, id string, created, updated time.Time) error {
if created.IsZero() && updated.IsZero() {
return nil
}
// COALESCE, so a note carrying only one of the two keeps what it already had
// for the other rather than being stamped with the zero time — which would sort
// it to the year one.
var c, u any
if !created.IsZero() {
c = created.Unix()
}
if !updated.IsZero() {
u = updated.Unix()
}
_, err := db.ExecContext(ctx, `
UPDATE notes
SET created_at = COALESCE(?, created_at),
updated_at = COALESCE(?, updated_at)
WHERE id = ?`, c, u, id)
return err
}
// SetNoteArchived puts a note away, or takes it back out.
//
// Separate from DeleteNote because they mean different things and the difference
// is the whole feature: the trash says "this was a mistake, and in thirty days it
// is gone", and the archive says "this is finished, and I want to keep it". A
// twelve-hundred-note import is mostly the second, and without somewhere to put
// them the list is the whole archive forever.
//
// `updated_at` is deliberately not touched. Archiving is not editing, and a list
// sorted by when things were last written should not be reordered by somebody
// tidying up.
func (db *DB) SetNoteArchived(ctx context.Context, id string, archived bool) error {
var at any
if archived {
at = time.Now().Unix()
}
_, err := db.ExecContext(ctx,
`UPDATE notes SET archived_at = ? WHERE id = ? AND deleted_at IS NULL`, at, id)
return err
}
// DeleteNote puts it in the trash. Nothing here removes a row.
func (db *DB) DeleteNote(ctx context.Context, id string) error {
_, err := db.ExecContext(ctx,
`UPDATE notes SET deleted_at = ?, updated_at = ? WHERE id = ? AND deleted_at IS NULL`,
time.Now().Unix(), time.Now().Unix(), id)
return err
}
// RestoreNote takes it back out.
func (db *DB) RestoreNote(ctx context.Context, id string) error {
_, err := db.ExecContext(ctx,
`UPDATE notes SET deleted_at = NULL, updated_at = ? WHERE id = ?`,
time.Now().Unix(), id)
return err
}
func (db *DB) notesWhere(ctx context.Context, where string, args ...any) ([]Note, error) {
return db.notesJoin(ctx, noteColumns, where, args...)
}
// notesJoin is notesWhere with the column list spelled out, for the one caller
// that joins another table and must therefore say which `id` it means.
func (db *DB) notesJoin(ctx context.Context, columns, where string, args ...any) ([]Note, error) {
rows, err := db.QueryContext(ctx, `SELECT `+columns+` FROM notes `+where, args...)
if err != nil {
return nil, err
}
defer rows.Close()
var out []Note
for rows.Next() {
n, err := scanNote(rows)
if err != nil {
return nil, err
}
out = append(out, n)
}
return out, rows.Err()
}
func (db *DB) linksOf(ctx context.Context, noteID string) ([]NoteLink, error) {
rows, err := db.QueryContext(ctx,
`SELECT kind, target_id FROM note_links WHERE note_id = ? ORDER BY kind, target_id`, noteID)
if err != nil {
return nil, err
}
defer rows.Close()
var out []NoteLink
for rows.Next() {
var l NoteLink
if err := rows.Scan(&l.Kind, &l.TargetID); err != nil {
return nil, err
}
out = append(out, l)
}
return out, rows.Err()
}
func scanNote(row scanner) (Note, error) {
var n Note
var projectID, createdBy sql.NullString
var created, updated int64
var deleted, archived sql.NullInt64
if err := row.Scan(&n.ID, &projectID, &n.Title, &n.Body, &createdBy,
&n.Pinned, &created, &updated, &deleted, &archived); err != nil {
return n, err
}
n.ProjectID = projectID.String
n.CreatedBy = createdBy.String
n.CreatedAt = time.Unix(created, 0)
n.UpdatedAt = time.Unix(updated, 0)
if deleted.Valid {
t := time.Unix(deleted.Int64, 0)
n.DeletedAt = &t
}
if archived.Valid {
t := time.Unix(archived.Int64, 0)
n.ArchivedAt = &t
}
return n, nil
}
+371
View File
@@ -0,0 +1,371 @@
package store
import (
"context"
"strings"
"testing"
)
// The point of the whole thing: a #tag in a note is the same tag as in a task.
// Not a second kind that looks alike and has to be kept in step by hand.
func TestATagInANoteIsTheSameTagAsInATask(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
n := &Note{
CreatedBy: userID,
Body: "Mødet med #Firma gik fint. Følg op sammen med #Regnskab.",
}
if err := db.SaveNote(ctx, n); err != nil {
t.Fatal(err)
}
var projects []string
for _, l := range n.Links {
if l.Kind == "project" {
projects = append(projects, l.TargetID)
}
}
if len(projects) != 2 {
t.Fatalf("read %v out of the note, want the two projects", n.Links)
}
// Folded: the key is a key, not a label. #firma and #Firma are the same project
// to everybody except a database, and the project's own spelling is on the
// project, which is what the interface shows.
if projects[0] != "firma" || projects[1] != "regnskab" {
t.Errorf("got %v", projects)
}
}
// Backwards is the direction that makes it a second brain rather than a folder.
func TestANoteIsFoundFromWhatItPointsAt(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
const taskID = "01a01694-a25c-7000-a50f-d72ab33f9b6a"
notes := []*Note{
{CreatedBy: userID, Body: "Aftalt på mødet, se /opgave/" + taskID},
{CreatedBy: userID, Body: "Intet at gøre med den sag."},
}
for _, n := range notes {
if err := db.SaveNote(ctx, n); err != nil {
t.Fatal(err)
}
}
found, err := db.NotesLinking(ctx, "task", taskID)
if err != nil {
t.Fatal(err)
}
if len(found) != 1 {
t.Fatalf("%d notes point at the task, want 1", len(found))
}
if !strings.HasPrefix(found[0].Title, "Aftalt") {
t.Errorf("the wrong note came back: %q", found[0].Title)
}
}
// The links are an index over the text, so editing the text must move them.
// A stale link is worse than none: it says a connection exists that does not.
func TestEditingANoteRewritesWhatItPointsAt(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
n := &Note{CreatedBy: userID, Body: "Handler om #Firma"}
if err := db.SaveNote(ctx, n); err != nil {
t.Fatal(err)
}
if before, _ := db.NotesLinking(ctx, "project", "Firma"); len(before) != 1 {
t.Fatal("the first save did not record the link")
}
n.Body = "Handler om #Regnskab i stedet"
if err := db.SaveNote(ctx, n); err != nil {
t.Fatal(err)
}
stale, err := db.NotesLinking(ctx, "project", "Firma")
if err != nil {
t.Fatal(err)
}
if len(stale) != 0 {
t.Errorf("the note still points at Firma after the mention was removed")
}
if fresh, _ := db.NotesLinking(ctx, "project", "Regnskab"); len(fresh) != 1 {
t.Errorf("the new mention was not recorded")
}
}
// A list of untitled notes is a list nobody can read.
func TestANoteTakesItsTitleFromItsFirstLine(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
for _, c := range []struct{ body, want string }{
{"# Møde med Anders\n\nHan ville gerne...", "Møde med Anders"},
{"\n\n Løse tanker \nmere", "Løse tanker"},
{"", ""},
} {
n := &Note{CreatedBy: userID, Body: c.body}
if err := db.SaveNote(ctx, n); err != nil {
t.Fatal(err)
}
if n.Title != c.want {
t.Errorf("body %q gave title %q, want %q", c.body, n.Title, c.want)
}
}
}
// Danish spelling, both ways round — the same generosity tasks already get.
func TestNotesAreFoundInEitherSpelling(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
// The title is the first line now, so that is where it goes.
if err := db.SaveNote(ctx, &Note{
CreatedBy: userID,
Body: "Grønt regnskab\n\nMålt på årsbasis for Århus.",
}); err != nil {
t.Fatal(err)
}
for _, q := range []string{"grønt", "gront", "århus", "aarhus", "regnskab"} {
found, err := db.SearchNotes(ctx, q, 10)
if err != nil {
t.Fatal(err)
}
if len(found) != 1 {
t.Errorf("%q found %d notes, want 1", q, len(found))
}
}
}
// Deleting is the trash, the way it is everywhere else in this program. A note is
// the one place somebody can lose an hour of writing to one keystroke.
func TestADeletedNoteCanComeBack(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
n := &Note{CreatedBy: userID, Body: "Vigtigt"}
if err := db.SaveNote(ctx, n); err != nil {
t.Fatal(err)
}
if err := db.DeleteNote(ctx, n.ID); err != nil {
t.Fatal(err)
}
if gone, _ := db.Note(ctx, n.ID); gone != nil {
t.Error("a deleted note is still returned")
}
if err := db.RestoreNote(ctx, n.ID); err != nil {
t.Fatal(err)
}
back, err := db.Note(ctx, n.ID)
if err != nil {
t.Fatal(err)
}
if back == nil || back.Body != "Vigtigt" {
t.Error("the note did not come back whole")
}
}
func TestWhatIsReadOutOfABody(t *testing.T) {
for _, c := range []struct {
name, body string
want []NoteLink
}{
{"a hash inside a word is not a tag", "C#Sharp og x#y", nil},
{"a note by title", "se [[Møde med Anders]]", []NoteLink{{"note", "Møde med Anders"}}},
{"the same tag twice is one link", "#Firma og #Firma igen",
[]NoteLink{{"project", "firma"}}},
{"two spellings are one tag", "#Firma og #firma",
[]NoteLink{{"project", "firma"}}},
{"a heading is not a tag", "# Overskrift", nil},
} {
t.Run(c.name, func(t *testing.T) {
got := LinksIn(c.body)
if len(got) != len(c.want) {
t.Fatalf("got %v, want %v", got, c.want)
}
for i := range got {
if got[i] != c.want[i] {
t.Errorf("got %v, want %v", got[i], c.want[i])
}
}
})
}
}
// The number beside a project is what is left to do: not finished, not deleted.
// It replaced a count of people, which said 2 on an empty project.
func TestTheProjectCountIsWhatIsLeft(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
p := &Project{Name: "Tallene", OwnerID: userID}
if err := db.CreateProject(ctx, p); err != nil {
t.Fatal(err)
}
count := func() int {
list, err := db.ListProjects(ctx, userID, false)
if err != nil {
t.Fatal(err)
}
for _, x := range list {
if x.ID == p.ID {
return x.OpenCount
}
}
t.Fatal("the project is not in the list")
return -1
}
if n := count(); n != 0 {
t.Errorf("an empty project counts %d", n)
}
var ids []string
for _, c := range []string{"en", "to", "tre"} {
task := &Task{ProjectID: p.ID, Content: c, CreatedBy: userID}
if err := db.CreateTask(ctx, task, nil); err != nil {
t.Fatal(err)
}
ids = append(ids, task.ID)
}
if n := count(); n != 3 {
t.Errorf("three tasks count %d", n)
}
if _, err := db.CompleteTask(ctx, ids[0], userID); err != nil {
t.Fatal(err)
}
if n := count(); n != 2 {
t.Errorf("after finishing one, the count is %d, want 2", n)
}
if err := db.DeleteTask(ctx, ids[1]); err != nil {
t.Fatal(err)
}
if n := count(); n != 1 {
t.Errorf("after deleting one, the count is %d, want 1", n)
}
}
// The title follows the first line, always — not only the first time.
//
// Derived once and then left alone, it goes stale the moment somebody rewrites
// the opening, and the list ends up calling a note by a name that is nowhere in
// it. That is exactly what happened: a note headed "Ny note" was listed as
// "fdgfgfgh", which was what its first line had been some edits ago.
func TestTheTitleFollowsTheFirstLine(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
n := &Note{CreatedBy: userID, Body: "Første udgave\n\nnoget tekst"}
if err := db.SaveNote(ctx, n); err != nil {
t.Fatal(err)
}
if n.Title != "Første udgave" {
t.Fatalf("title started as %q", n.Title)
}
n.Body = "# Ny note\n\nDette er mit nye noteprogram"
if err := db.SaveNote(ctx, n); err != nil {
t.Fatal(err)
}
if n.Title != "Ny note" {
t.Errorf("after rewriting the first line the title is %q", n.Title)
}
back, err := db.Note(ctx, n.ID)
if err != nil {
t.Fatal(err)
}
if back.Title != "Ny note" {
t.Errorf("the stored title is %q", back.Title)
}
// And a title cannot be set beside the text, because then the two can disagree.
n.Title = "noget andet"
if err := db.SaveNote(ctx, n); err != nil {
t.Fatal(err)
}
if n.Title != "Ny note" {
t.Errorf("a title set by hand survived as %q", n.Title)
}
}
// A note that names a project in the other case must still turn up on it.
//
// It did not, and nothing said why: the key was stored exactly as typed and
// looked up the same way, so #garageristeriet and #GarageRisteriet were two
// different things that looked identical to a person.
func TestATagFindsItsProjectWhateverTheCase(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
if err := db.SaveNote(ctx, &Note{
CreatedBy: userID,
Body: "Bønnerne kommer fredag #garageristeriet",
}); err != nil {
t.Fatal(err)
}
for _, spelling := range []string{"GarageRisteriet", "garageristeriet", "GARAGERISTERIET"} {
found, err := db.NotesLinking(ctx, "project", spelling)
if err != nil {
t.Fatal(err)
}
if len(found) != 1 {
t.Errorf("looking up %q found %d notes, want 1", spelling, len(found))
}
}
// A note title is a title, not a key: two notes called "Møde" and "møde" are
// two notes, and folding them together would merge things somebody kept apart.
if err := db.SaveNote(ctx, &Note{CreatedBy: userID, Body: "se [[Møde]]"}); err != nil {
t.Fatal(err)
}
if found, _ := db.NotesLinking(ctx, "note", "møde"); len(found) != 0 {
t.Error("a note title was folded")
}
if found, _ := db.NotesLinking(ctx, "note", "Møde"); len(found) != 1 {
t.Error("a note title did not match itself")
}
}
// En søgning skal svare med det, der passer bedst, øverst.
//
// Den sorterede efter `updated_at`, hvilket var usynligt med ti noter og
// ubrugeligt med tolv hundrede: de blev alle importeret samme aften, så datoen
// siger ingenting, og loftet skar så listen af ved halvtreds tilfældige rækker.
// At søge "Tesla" svarede med en note om nginx først og noten, der hedder "Tesla
// Model Y", bagefter — hvilket læses som en søgning, der ikke virker, fordi det
// fra brugerens plads er præcis dét.
func TestSearchAnswersWithTheBestMatchFirst(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
// Skrevet i den rækkefølge, der ville give det forkerte svar: den, der kun
// nævner ordet i kroppen, gemmes sidst og har derfor den nyeste dato.
for _, body := range []string{
"# Tesla Model Y\n\nBilen selv.\n",
"# Ladestandere\n\nNoter om Tesla og andre mærker.\n",
"# nginx proxy manager backup\n\nEn konfiguration. Tesla nævnes i forbifarten.\n",
} {
if err := db.SaveNote(ctx, &Note{CreatedBy: userID, Body: body}); err != nil {
t.Fatal(err)
}
}
found, err := db.SearchNotes(ctx, "Tesla", 10)
if err != nil {
t.Fatal(err)
}
if len(found) != 3 {
t.Fatalf("fandt %d noter, ventede 3", len(found))
}
if found[0].Title != "Tesla Model Y" {
t.Errorf("øverst stod %q; den note, der hedder det, man søgte efter, skal stå først", found[0].Title)
}
}
+31 -41
View File
@@ -140,11 +140,17 @@ func (db *DB) SavePushSubscription(ctx context.Context, sub *PushSubscription) e
if sub.ID == "" {
sub.ID = NewID()
}
// The upsert no longer moves the row between accounts.
//
// `user_id = excluded.user_id` was in the DO UPDATE, so anybody who learned
// another person's endpoint could claim it by subscribing with it: the
// victim's device would then receive the claimant's notifications and none of
// its own. A re-subscribe from the same person still refreshes the keys, which
// is what the upsert is actually for.
_, err := db.ExecContext(ctx,
`INSERT INTO push_subscriptions (id, user_id, endpoint, p256dh, auth, user_agent, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (endpoint) DO UPDATE SET
user_id = excluded.user_id,
p256dh = excluded.p256dh,
auth = excluded.auth,
user_agent = excluded.user_agent,
@@ -175,7 +181,21 @@ func (db *DB) ListPushSubscriptions(ctx context.Context, userID string) ([]PushS
return out, rows.Err()
}
func (db *DB) DeletePushSubscription(ctx context.Context, endpoint string) error {
// DeletePushSubscription removes one of this person's devices.
//
// Scoped by user as well as by endpoint. It was endpoint alone, which meant
// anybody who learned somebody else's endpoint could silently unsubscribe their
// phone — and the endpoint is the only thing about a subscription that ever
// leaves the browser.
func (db *DB) DeletePushSubscription(ctx context.Context, userID, endpoint string) error {
_, err := db.ExecContext(ctx,
`DELETE FROM push_subscriptions WHERE endpoint = ? AND user_id = ?`, endpoint, userID)
return err
}
// dropPushEndpoint removes a subscription the push service has declared dead.
// Unexported so the only *request* path to a delete stays the scoped one.
func (db *DB) dropPushEndpoint(ctx context.Context, endpoint string) error {
_, err := db.ExecContext(ctx, `DELETE FROM push_subscriptions WHERE endpoint = ?`, endpoint)
return err
}
@@ -186,7 +206,10 @@ func (db *DB) DeletePushSubscription(ctx context.Context, endpoint string) error
// silently unsubscribing somebody's phone.
func (db *DB) RecordPushFailure(ctx context.Context, endpoint string, permanent bool) error {
if permanent {
return db.DeletePushSubscription(ctx, endpoint)
// Not scoped by user, and that is right here: this is the push service
// saying the endpoint is gone, not a person asking for something. There is
// no caller to check against — the caller is the delivery job.
return db.dropPushEndpoint(ctx, endpoint)
}
_, err := db.ExecContext(ctx,
`UPDATE push_subscriptions SET failures = failures + 1 WHERE endpoint = ?`, endpoint)
@@ -272,44 +295,6 @@ func (db *DB) UserByMailToken(ctx context.Context, token string) (*User, error)
return db.scanUser(ctx, `SELECT `+userColumns+` FROM users WHERE mail_token = ?`, token)
}
// UsersWithGmail returns everybody who has connected a mailbox, which is who the
// poller has to visit. Selected by the presence of a refresh token rather than by a
// flag, so a disconnected account drops out of the sweep the moment its tokens are
// cleared.
func (db *DB) UsersWithGmail(ctx context.Context) ([]User, error) {
rows, err := db.QueryContext(ctx, `
SELECT u.id FROM users u
JOIN user_settings s ON s.user_id = u.id
WHERE s.scope = 'gmail' AND s.values_json LIKE '%"refresh_token":"%'`)
if err != nil {
return nil, err
}
var ids []string
for rows.Next() {
var id string
if err := rows.Scan(&id); err != nil {
rows.Close()
return nil, err
}
ids = append(ids, id)
}
err = rows.Err()
rows.Close()
if err != nil {
return nil, err
}
out := make([]User, 0, len(ids))
for _, id := range ids {
user, err := db.UserByID(ctx, id)
if err != nil {
continue
}
out = append(out, *user)
}
return out, nil
}
// --- per-user settings ------------------------------------------------------------------
// UserSettings reads one scope of a person's integration settings. A missing row is
@@ -327,6 +312,11 @@ var secretFields = map[string]bool{
"access_token": true,
"password": true,
"client_secret": true,
// The AI provider's key. Missed when this list was written, and it is exactly
// the same kind of thing as the rest: a live credential, in a settings blob,
// in a database anybody with a backup file can read. Existing rows convert
// themselves the next time the settings are saved.
"api_key": true,
}
// seal returns a copy of values with the secret fields sealed. Without a key it
+16 -1
View File
@@ -30,6 +30,11 @@ type Project struct {
Role Role
// MemberCount is how many people can see it; 1 means it is not shared.
MemberCount int
// OpenCount is what is left to do: not deleted, not finished. It is the number
// the sidebar shows, because a number beside a project in a task app is read as
// tasks whatever it was meant to be.
OpenCount int
}
type Section struct {
@@ -50,6 +55,13 @@ func (db *DB) ListProjects(ctx context.Context, userID string, includeArchived b
p.archived, p.sort_order, p.created_at, p.updated_at,
CASE WHEN p.owner_id = ? THEN 'owner' ELSE COALESCE(m.role, '') END AS role,
(SELECT count(*) FROM project_members pm WHERE pm.project_id = p.id) + 1 AS member_count,
-- What is left to do, which is what the number beside a project in the
-- sidebar means to the person reading it. It used to show member_count,
-- and a "2" on an empty project read as two tasks to everybody who saw
-- it — including the person who wrote the app.
(SELECT count(*) FROM tasks t
WHERE t.project_id = p.id AND t.deleted_at IS NULL AND t.completed_at IS NULL
) AS open_count,
CASE WHEN p.owner_id = ? THEN p.group_id END AS group_id
FROM projects p
LEFT JOIN project_members m ON m.project_id = p.id AND m.user_id = ?
@@ -88,6 +100,9 @@ func (db *DB) GetProject(ctx context.Context, projectID, userID string) (*Projec
p.archived, p.sort_order, p.created_at, p.updated_at,
? AS role,
(SELECT count(*) FROM project_members pm WHERE pm.project_id = p.id) + 1,
(SELECT count(*) FROM tasks t
WHERE t.project_id = p.id AND t.deleted_at IS NULL AND t.completed_at IS NULL
),
CASE WHEN p.owner_id = ? THEN p.group_id END
FROM projects p WHERE p.id = ? AND p.deleted_at IS NULL`, string(role), userID, projectID)
if err != nil {
@@ -113,7 +128,7 @@ func scanProject(rows *sql.Rows) (Project, error) {
var role string
err := rows.Scan(&p.ID, &p.Name, &p.Color, &icon, &p.ViewMode, &p.OwnerID, &isInbox,
&archived, &p.SortOrder, &created, &updated, &role, &p.MemberCount, &groupID)
&archived, &p.SortOrder, &created, &updated, &role, &p.MemberCount, &p.OpenCount, &groupID)
if err != nil {
return p, err
}
+40
View File
@@ -0,0 +1,40 @@
package store
import (
"context"
"testing"
)
// Saving a mailbox that is already there must update it, not fail.
//
// The row is written again on every token refresh and every time Gmail's seen
// list changes. If that write errors, the tasks a sync made are kept and the note
// of having made them is not — so the same mail becomes a task again on the next
// sweep, once every ten minutes, forever.
func TestSavingAnExistingMailboxUpdatesIt(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
m := &Mailbox{UserID: userID, Kind: "gmail", Username: "[email protected]",
RefreshToken: "r", Seen: []string{"a"}}
if err := db.SaveMailbox(ctx, m); err != nil {
t.Fatal(err)
}
m.Seen = []string{"a", "b"}
m.AccessToken = "fresh"
if err := db.SaveMailbox(ctx, m); err != nil {
t.Fatalf("saving it a second time failed: %v", err)
}
back, err := db.MailboxOfKind(ctx, userID, "gmail")
if err != nil {
t.Fatal(err)
}
if len(back.Seen) != 2 {
t.Errorf("the seen list came back as %v", back.Seen)
}
if back.AccessToken != "fresh" {
t.Errorf("the token came back as %q", back.AccessToken)
}
}
+41 -2
View File
@@ -20,6 +20,13 @@ func FoldDanish(s string) string {
return r.Replace(s)
}
// The ceilings on a search. See MatchExprOver for why they exist; both are far
// above anything a person types and far below anything that costs.
const (
maxSearchInput = 512
maxSearchTerms = 16
)
// MatchExpr turns what someone typed into the search box into an FTS5 MATCH
// expression.
//
@@ -36,18 +43,45 @@ func FoldDanish(s string) string {
// Returns "" when there is nothing searchable in the input; callers should treat
// that as "no query" rather than as "match everything".
func MatchExpr(query string) string {
return MatchExprOver(query, "content", "description")
}
// MatchExprOver is MatchExpr against a named set of columns.
//
// The columns have to be named because the index they are searched in decides
// them: tasks carry content and description, notes carry title and body. Passing
// one table's column names to the other's index is not a wrong answer but an
// error — "no such column" — which is at least loud.
func MatchExprOver(query string, columns ...string) string {
terms := tokenize(query)
if len(terms) == 0 {
if len(terms) == 0 || len(columns) == 0 {
return ""
}
// Capped, because the expression is built from the input and then run inside
// SQLite on one of a handful of pooled connections.
//
// Every term becomes roughly fifty bytes of FTS5, and every one of them is a
// prefix query — which is what makes ⌘K feel like it searches as you type, and
// also what makes each one expensive. A quarter of a megabyte pasted into the
// box became a two-megabyte MATCH expression and half a minute inside SQLite,
// holding a connection the whole time. A few of those at once and the database
// is unavailable to everybody on the instance, logins included.
//
// Sixteen terms is far past anything a person types into a search box and
// nowhere near enough to hurt. Silently, because a message about a limit
// nobody has reached on purpose is noise.
if len(terms) > maxSearchTerms {
terms = terms[:maxSearchTerms]
}
var b strings.Builder
for i, term := range terms {
if i > 0 {
b.WriteString(" AND ")
}
folded := FoldDanish(term)
b.WriteString(`({content description} : `)
b.WriteString(`({` + strings.Join(columns, " ") + `} : `)
b.WriteString(quote(term))
b.WriteString(`* OR fold : `)
b.WriteString(quote(folded))
@@ -60,6 +94,11 @@ func MatchExpr(query string) string {
// punctuation rather than whitespace alone means "q3/2026" searches for "q3" and
// "2026" instead of for a term FTS5's own tokenizer would never have indexed.
func tokenize(query string) []string {
// Trimmed before it is split, so a megabyte of text is not walked rune by rune
// only to have all but sixteen of the results thrown away.
if len(query) > maxSearchInput {
query = query[:maxSearchInput]
}
fields := strings.FieldsFunc(query, func(r rune) bool {
return !unicode.IsLetter(r) && !unicode.IsDigit(r)
})
+7 -6
View File
@@ -84,22 +84,23 @@ func TestATokenWrittenBeforeTheKeyStillWorks(t *testing.T) {
}
}
// UsersWithGmail finds people by the presence of the key, not its value, so it
// must keep working now that the value is ciphertext.
// The sweep finds people by the presence of a mailbox, so it must keep working
// now that the credential in it is ciphertext.
func TestTheSweepStillFindsAConnectedMailbox(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
if err := db.SetUserSettings(ctx, userID, "gmail",
map[string]any{"refresh_token": "1//0-token"}); err != nil {
if err := db.SaveMailbox(ctx, &Mailbox{
UserID: userID, Kind: "gmail", RefreshToken: "1//0-token",
}); err != nil {
t.Fatal(err)
}
users, err := db.UsersWithGmail(ctx)
users, err := db.UsersWithMailboxes(ctx)
if err != nil {
t.Fatal(err)
}
if len(users) != 1 {
t.Fatalf("the sweep found %d connected mailboxes, want 1", len(users))
t.Fatalf("the sweep found %d people with mailboxes, want 1", len(users))
}
}
+53 -7
View File
@@ -36,8 +36,13 @@ type User struct {
// is as true on the laptop as on the desktop. The sidebar's *width* is the
// opposite case and is stored per browser.
SidebarCollapsed []string
CreatedAt time.Time
UpdatedAt time.Time
// NavOrder is the order this person wants the fixed views in, by key. Empty
// means the order the program ships with. Unknown keys are ignored and missing
// ones are appended, so adding a view later is not a migration.
NavOrder []string
CreatedAt time.Time
UpdatedAt time.Time
}
// NormalizeEmail is the one place an address is canonicalised. Addresses arrive
@@ -104,7 +109,7 @@ func (db *DB) UserByID(ctx context.Context, id string) (*User, error) {
}
const userColumns = `id, email, name, password_hash, totp_secret, totp_enabled,
avatar_color, timezone, locale, is_admin, sidebar_collapsed, created_at, updated_at`
avatar_color, timezone, locale, is_admin, sidebar_collapsed, nav_order, created_at, updated_at`
func (db *DB) scanUser(ctx context.Context, query string, args ...any) (*User, error) {
var u User
@@ -112,11 +117,11 @@ func (db *DB) scanUser(ctx context.Context, query string, args ...any) (*User, e
var totpEnabled, isAdmin int
var created, updated int64
var collapsed string
var collapsed, navOrder string
err := db.QueryRowContext(ctx, query, args...).Scan(
&u.ID, &u.Email, &u.Name, &u.PasswordHash, &secret, &totpEnabled,
&u.AvatarColor, &u.Timezone, &u.Locale, &isAdmin, &collapsed, &created, &updated)
&u.AvatarColor, &u.Timezone, &u.Locale, &isAdmin, &collapsed, &navOrder, &created, &updated)
if errors.Is(err, sql.ErrNoRows) {
return nil, ErrNotFound
}
@@ -124,18 +129,48 @@ func (db *DB) scanUser(ctx context.Context, query string, args ...any) (*User, e
return nil, err
}
u.TOTPSecret = secret.String
// Opened on the way out. A row written before the key existed is returned as
// itself — see secret.Unseal — so this does not need a migration; a secret
// converts the next time it is written.
if plain, err := db.unsealValue(secret.String); err == nil {
u.TOTPSecret = plain
} else {
u.TOTPSecret = secret.String
}
u.TOTPEnabled = totpEnabled == 1
u.IsAdmin = isAdmin == 1
// A malformed value is read as "nothing folded" rather than as an error: the
// worst case is a sidebar that opens fully, which is the state it shipped in.
u.SidebarCollapsed = []string{}
_ = json.Unmarshal([]byte(collapsed), &u.SidebarCollapsed)
u.NavOrder = []string{}
_ = json.Unmarshal([]byte(navOrder), &u.NavOrder)
u.CreatedAt = time.Unix(created, 0).UTC()
u.UpdatedAt = time.Unix(updated, 0).UTC()
return &u, nil
}
// SetNavOrder records the order of the fixed views.
func (db *DB) SetNavOrder(ctx context.Context, userID string, order []string) error {
if order == nil {
order = []string{}
}
raw, err := json.Marshal(order)
if err != nil {
return err
}
res, err := db.ExecContext(ctx,
`UPDATE users SET nav_order = ?, updated_at = ? WHERE id = ?`,
string(raw), time.Now().Unix(), userID)
if err != nil {
return err
}
if n, _ := res.RowsAffected(); n == 0 {
return ErrNotFound
}
return nil
}
// SetSidebarCollapsed records which sidebar headings are folded.
//
// Its own method rather than a field on UpdateProfile: that one backs a form with
@@ -405,9 +440,20 @@ func (db *DB) UpdateProfile(ctx context.Context, userID, name, timezone, locale
}
func (db *DB) SetTOTPSecret(ctx context.Context, userID, secret string, enabled bool) error {
// Sealed, like a mailbox password and for the same reason: the copies, not the
// server. A TOTP secret in a downloadable backup lets whoever holds the copy
// generate valid codes for every account on the instance — which turns the
// second factor from a defence into a file somebody can take home.
//
// It was missed when internal/secret was written, along with three other
// values; see SECURITY.md.
var value any
if secret != "" {
value = secret
sealed, err := db.sealValue(secret)
if err != nil {
return err
}
value = sealed
}
_, err := db.ExecContext(ctx,
`UPDATE users SET totp_secret = ?, totp_enabled = ?, updated_at = ? WHERE id = ?`,
+2
View File
@@ -48,12 +48,14 @@ nav:
- As a Rune: rune.md
- Configuration: configuration.md
- Using it:
- Signing in: signing-in.md
- Quick add: quick-add.md
- Filters: filters.md
- Importing from Todoist: import.md
- Connecting things:
- Calendar and CalDAV: caldav.md
- Mail to task: mail.md
- Mailboxes: mailboxes.md
- Claude (MCP): mcp.md
- AI features: ai.md
# openapi.yaml is deliberately not in the nav: MkDocs copies it to the site as a
+296 -17
View File
@@ -6,13 +6,18 @@
<title>verdande — self-hosted tasks, shared</title>
<meta
name="description"
content="Self-hosted tasks and projects, shared. One Go binary with SQLite inside it. CalDAV, mail-to-task and an MCP connector for Claude."
content="Self-hosted tasks and projects, shared. One Go binary with SQLite inside it. Your mail as an inbox, CalDAV, passkeys, and an MCP connector for Claude."
/>
<meta name="theme-color" content="#111312" />
<meta property="og:title" content="verdande" />
<meta property="og:description" content="Self-hosted tasks and projects, shared. One binary, SQLite inside it." />
<meta property="og:type" content="website" />
<meta property="og:image" content="screenshots/today.webp" />
<meta property="og:image:alt" content="The Today view: projects and filters beside a list of what is due." />
<!-- Summary rather than the large card: the screenshot is a dense interface,
and shrunk into a wide banner it reads as grey noise. -->
<meta name="twitter:card" content="summary" />
<link rel="icon" href="icon.svg" type="image/svg+xml" />
@@ -38,6 +43,10 @@
--font-mono: ui-monospace, 'SF Mono', 'JetBrains Mono', Menlo, monospace;
}
/* Written twice on purpose. The media query is what a first-time visitor
gets; the attribute is what somebody who has chosen gets, and it has to
win in both directions — a chosen light theme on a dark system as well
as the other way round. A single rule cannot do both. */
@media (prefers-color-scheme: light) {
:root:not([data-theme='dark']) {
--ground: #faf9f7;
@@ -53,6 +62,63 @@
}
}
:root[data-theme='light'] {
--ground: #faf9f7;
--surface: #ffffff;
--surface-raised: #ffffff;
--line: #e5e3de;
--line-strong: #d3d0c9;
--ink: #1c201e;
--ink-muted: #5f6763;
--ink-faint: #8b918d;
--accent: #2f8f63;
--accent-ink: #ffffff;
}
.theme-toggle {
position: fixed;
top: 1rem;
right: 1rem;
z-index: 10;
width: 2.25rem;
height: 2.25rem;
display: grid;
place-items: center;
border: 1px solid var(--line);
border-radius: 999px;
background: var(--surface);
color: var(--ink-muted);
cursor: pointer;
font-size: 1rem;
line-height: 1;
}
.theme-toggle:hover {
color: var(--ink);
border-color: var(--line-strong);
}
/* Whichever screenshot does not belong to the theme in force is simply not
there. Both are in the markup so neither has to be fetched at the moment
somebody flips the switch. */
:root:not([data-theme='light']) .only-light,
:root[data-theme='dark'] .only-light {
display: none;
}
:root[data-theme='light'] .only-dark {
display: none;
}
@media (prefers-color-scheme: light) {
:root:not([data-theme='dark']) .only-light {
display: block;
}
:root:not([data-theme='dark']) .only-dark {
display: none;
}
}
*,
*::before,
*::after {
@@ -339,6 +405,89 @@
text-align: center;
}
/* --- zoom -------------------------------------------------------------
A screenshot of an interface is unreadable at 280px in a three-column
grid, which is where most of them sit. Clicking one opens it at the
size it was taken.
<dialog> rather than a hand-built overlay: Escape closes it, focus is
trapped, the rest of the page is inert, and the browser does all three
without a line of script. */
.shot img {
cursor: zoom-in;
}
/* Sized so the whole thing fits: a screenshot is 1.55 times wider than it
is tall, so on a laptop the height is what runs out first. Left to grow,
it pushed the caption and the close button off the top of the screen —
and the button is the visible way out. */
dialog.zoom {
max-width: 96vw;
max-height: 100vh;
/* Wraps the image rather than the viewport. With a plain `auto` the box
stayed full width, so the screenshot sat against the left edge and
the close button ended up in the far corner, nowhere near it. */
width: fit-content;
margin: auto;
padding: 2.5rem 0 0;
border: 0;
background: transparent;
overflow: visible;
}
dialog.zoom::backdrop {
background: rgb(0 0 0 / 0.82);
}
dialog.zoom img {
max-width: 96vw;
max-height: 78vh;
width: auto;
height: auto;
display: block;
border-radius: var(--radius-lg);
cursor: zoom-out;
}
dialog.zoom figcaption {
margin-top: 0.75rem;
font-size: 0.8125rem;
color: #a8b0ac;
text-align: center;
}
/* The whole backdrop closes, so the button is a visible affordance rather
than the only way out. */
dialog.zoom .close {
position: absolute;
top: 0;
right: 0;
background: none;
border: 0;
color: #a8b0ac;
font-size: 1.5rem;
line-height: 1;
cursor: pointer;
padding: 0.25rem 0.5rem;
}
dialog.zoom .close:hover {
color: #fff;
}
@media (prefers-reduced-motion: no-preference) {
dialog.zoom[open] {
animation: zoom-in 0.15s ease-out;
}
@keyframes zoom-in {
from {
opacity: 0;
transform: scale(0.98);
}
}
}
.shots {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(280px, 1fr));
@@ -375,6 +524,9 @@
</head>
<body>
<button class="theme-toggle" type="button" aria-label="Switch between light and dark" title="Light or dark">
<span aria-hidden="true">◐</span>
</button>
<div class="wrap">
<header>
<span class="mark" aria-hidden="true">ᚹ</span>
@@ -434,9 +586,9 @@
<section>
<h2>What it looks like</h2>
<figure class="shot">
<figure class="shot only-dark">
<img
src="screenshots/today.png"
src="screenshots/today.webp"
alt="The Today view: a sidebar of projects, filters and labels beside a list of tasks due today, each with its priority ring, time and labels."
width="1400"
height="900"
@@ -445,10 +597,21 @@
<figcaption>Today. Dark is the default; there is a light theme too.</figcaption>
</figure>
<figure class="shot only-light">
<img
src="screenshots/today-light.webp"
alt="The Today view in the light theme: a sidebar of projects, filters and labels beside a list of tasks due today."
width="1400"
height="900"
loading="lazy"
/>
<figcaption>Today. Dark is the default; there is a light theme too.</figcaption>
</figure>
<div class="shots">
<figure class="shot">
<img
src="screenshots/board.png"
src="screenshots/board.webp"
alt="A project in board view: one column per section, with task cards showing due dates, labels and repetition."
width="1400"
height="900"
@@ -458,17 +621,17 @@
</figure>
<figure class="shot">
<img
src="screenshots/today-light.png"
alt="The same Today view in the light theme."
src="screenshots/notes.webp"
alt="Notes: a list of notes on the left, one open on the right with headings, bullet lists, a quotation and a shell command in a code block."
width="1400"
height="900"
loading="lazy"
/>
<figcaption>The light theme.</figcaption>
<figcaption>Notes. Rich text on top, Markdown underneath.</figcaption>
</figure>
<figure class="shot">
<img
src="screenshots/week.png"
src="screenshots/week.webp"
alt="Upcoming as a single week: seven columns of days, each holding the tasks due that day."
width="1400"
height="900"
@@ -478,8 +641,8 @@
</figure>
<figure class="shot">
<img
src="screenshots/detail.png"
alt="A task open in the detail drawer, with its description, due date, labels, sub-tasks and comments."
src="screenshots/detail.webp"
alt="A task open in the detail drawer, with its description, priority, due date, labels, sub-tasks and an attached file."
width="1400"
height="900"
loading="lazy"
@@ -488,17 +651,17 @@
</figure>
<figure class="shot">
<img
src="screenshots/themes.png"
alt="The settings page showing the five themes as swatches."
src="screenshots/themes.webp"
alt="The settings page showing five themes as swatches and, below them, four looks written in the typeface each one selects."
width="1400"
height="900"
loading="lazy"
/>
<figcaption>Five themes, all of them measured for contrast.</figcaption>
<figcaption>Five themes and four looks: how bright, and how it reads.</figcaption>
</figure>
<figure class="shot">
<img
src="screenshots/history.png"
src="screenshots/history.webp"
alt="The instance-wide history page: who did what, in which project, filtered by person and event."
width="1400"
height="900"
@@ -526,6 +689,16 @@
invite link. Changes appear live.
</p>
</div>
<div class="cell">
<h3>Notes, beside the work</h3>
<p>
Rich text to write in, Markdown on disk. File a note in a project and
everybody in it can read it; <code>#project</code> and
<code>[[another note]]</code> are links, and a task shows what has been
written about it. A zip of Markdown goes in and comes back out the same
shape, so nothing is trapped here.
</p>
</div>
<div class="cell">
<h3>A real CalDAV server</h3>
<p>
@@ -541,10 +714,20 @@
</p>
</div>
<div class="cell">
<h3>Mail to task</h3>
<h3>Your mail, as an inbox</h3>
<p>
Forward an email to your own address and it becomes a task, subject
parsed like anything else you type.
Star a mail in Gmail, or flag one in iCloud or Fastmail, and it becomes
a task. Forward one to your own address instead and the subject is
parsed like anything else you type. One way: unstarring it later does
nothing to the task you made.
</p>
</div>
<div class="cell">
<h3>Nothing to remember</h3>
<p>
Start typing anywhere and you are writing a task, with the letter you
pressed already in the field. Sign in with a passkey and there is no
password to forget either.
</p>
</div>
<div class="cell">
@@ -554,6 +737,22 @@
round trip is tested, not promised.
</p>
</div>
<div class="cell">
<h3>Keys stay behind</h3>
<p>
Mail tokens and mailbox passwords are encrypted in the database, and the
key lives in a file beside it — not inside it. So a backup you download,
or one that ends up somewhere it should not, does not open your mail.
</p>
</div>
<div class="cell">
<h3>Who did what</h3>
<p>
Every change is written down — who moved it, who closed it, who let
somebody in. Delete the account and the trail stays: it is the
instance's record, not the person's.
</p>
</div>
</div>
</section>
@@ -605,5 +804,85 @@
</p>
</footer>
</div>
<!-- Zoom. Added by script rather than written into the markup, so a page with
no JavaScript is exactly what it was before: the screenshots are still
there, still readable, just not clickable. -->
<dialog class="zoom" aria-label="Screenshot, enlarged">
<button class="close" type="button" aria-label="Close">&times;</button>
<img alt="" />
<figcaption></figcaption>
</dialog>
<script>
// Runs before the zoom script and before anything is painted below it, so
// a chosen theme does not flash the other one first.
(function () {
var root = document.documentElement;
var stored = null;
try {
stored = localStorage.getItem('verdande-site-theme');
} catch (e) {
// Private browsing. The choice simply will not be remembered.
}
if (stored === 'light' || stored === 'dark') root.setAttribute('data-theme', stored);
document.querySelector('.theme-toggle').addEventListener('click', function () {
// What is on the screen right now, whether it came from a choice or
// from the system — otherwise the first press does nothing visible
// for somebody whose system already says light.
var showing =
root.getAttribute('data-theme') ||
(matchMedia('(prefers-color-scheme: light)').matches ? 'light' : 'dark');
var next = showing === 'light' ? 'dark' : 'light';
root.setAttribute('data-theme', next);
try {
localStorage.setItem('verdande-site-theme', next);
} catch (e) {}
});
})();
(function () {
var dialog = document.querySelector('dialog.zoom');
if (!dialog || typeof dialog.showModal !== 'function') return;
var big = dialog.querySelector('img');
var caption = dialog.querySelector('figcaption');
function open(img) {
big.src = img.currentSrc || img.src;
big.alt = img.alt;
var fig = img.closest('figure');
var text = fig && fig.querySelector('figcaption');
caption.textContent = text ? text.textContent : '';
dialog.showModal();
}
document.querySelectorAll('.shot img').forEach(function (img) {
// Focusable and operable from the keyboard: a control that only a
// mouse can reach is not a control everybody has.
img.tabIndex = 0;
img.setAttribute('role', 'button');
img.addEventListener('click', function () { open(img); });
img.addEventListener('keydown', function (event) {
if (event.key === 'Enter' || event.key === ' ') {
event.preventDefault();
open(img);
}
});
});
dialog.addEventListener('click', function (event) {
// The backdrop is the dialog itself; anything inside it is not.
if (event.target === dialog || event.target === big ||
event.target.classList.contains('close')) {
dialog.close();
}
});
// Let go of the image when it closes, so a large screenshot is not
// held in memory for the rest of the visit.
dialog.addEventListener('close', function () { big.removeAttribute('src'); });
})();
</script>
</body>
</html>
Binary file not shown.

Before

Width:  |  Height:  |  Size: 166 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 40 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 181 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 44 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 91 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 22 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 228 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 47 KiB

Some files were not shown because too many files have changed in this diff Show More