Compare commits
@@ -66,3 +66,6 @@ node_modules.nosync/
|
||||
# Kept on disk because it is how a session hands over to the next one.
|
||||
HANDOFF.md
|
||||
NOTES-PLAN.md
|
||||
|
||||
# Agenternes arbejdstræer. Et git-repo inde i et git-repo, som ikke hører til her.
|
||||
.claude/worktrees/
|
||||
|
||||
@@ -0,0 +1,437 @@
|
||||
# Notes, as verdande builds them
|
||||
|
||||
A description of one working notes system: the data model, the syntax, the search,
|
||||
the rendering contract, the import and export formats, and — more useful than any
|
||||
of it — the decisions that look arbitrary until you know what they cost.
|
||||
|
||||
It is written to be lifted into another program. Nothing here depends on Go,
|
||||
SQLite or Svelte except where it says so; where an answer was forced by the tool,
|
||||
that is called out so you can pick a different one on purpose rather than by
|
||||
accident.
|
||||
|
||||
---
|
||||
|
||||
## Read this part first: what to ask before you build
|
||||
|
||||
**This document is not a drop-in. It describes a notes system that lives inside a
|
||||
task app with projects, roles, a trash and a search box, and about a third of its
|
||||
design is inherited from that.** Lifting it into a program with different
|
||||
surroundings without deciding which parts to keep produces something that works
|
||||
for a week and then fights you.
|
||||
|
||||
**If you are an assistant working from this file: ask these questions and get
|
||||
answers before writing code.** Do not guess a default. Each one changes what you
|
||||
build, not just how you build it.
|
||||
|
||||
### About the surroundings
|
||||
|
||||
1. **Does the host app already have a place to file things** — projects, folders,
|
||||
notebooks, tags? verdande deliberately gave notes no hierarchy of their own and
|
||||
filed them under the projects that already existed. If your app has nothing,
|
||||
you need an answer for where a note lives, and "a folder tree" is the expensive
|
||||
one.
|
||||
2. **Is there more than one person?** Sharing, roles and "who wrote this" run
|
||||
through the whole model below. Single-user changes the schema and removes about
|
||||
half the permission surface.
|
||||
3. **Is there already a trash, and does it expire?** Notes lean on one. Without
|
||||
it, deleting is either permanent or you are building a trash.
|
||||
4. **Is there an existing full-text index?** The search here is FTS5-shaped. On
|
||||
Postgres it is `tsvector`, on a client-side app it is probably a scan, and the
|
||||
ranking section below changes accordingly.
|
||||
|
||||
### About the notes themselves
|
||||
|
||||
5. **Markdown, rich text, or blocks?** verdande stores Markdown and renders it.
|
||||
That decision is defended below; it is also the one that most constrains
|
||||
everything else.
|
||||
6. **Is the first line the title, or is there a title field?** Both work. Deriving
|
||||
it means one less thing to keep in step and one less thing a person can leave
|
||||
blank, and it forces the rule "the first line is a heading" on the editor.
|
||||
7. **What links to what?** `#project`, `[[note]]` and task references exist here
|
||||
because the app has projects and tasks. Yours may have none of them, or others.
|
||||
8. **Do notes carry files?** Attachments bring content-type handling, a store, an
|
||||
authorisation path and an XSS surface. Skipping them is a legitimate answer.
|
||||
|
||||
### About scale and language
|
||||
|
||||
9. **How many notes, realistically?** Several choices below flip at around ten
|
||||
thousand: the list payload, the search fallback, rendering the whole list.
|
||||
10. **Which languages?** The folding described under *Search* is Danish. Every
|
||||
language has its own version of that problem and the generic answer (strip
|
||||
diacritics) is wrong for several of them.
|
||||
11. **Where does the text end up besides the screen?** Search indexes, AI
|
||||
features, exports and backups each read it. Plaintext makes all of them cheap
|
||||
and makes a stolen backup expensive — see *What is still open*.
|
||||
|
||||
### About integration, specifically
|
||||
|
||||
12. **Who owns the note's identity — you or the other system?** If notes will sync
|
||||
with something else, ids and timestamps are the whole problem, and the
|
||||
`updated_at` rule below becomes the first thing you have to change.
|
||||
13. **Is this a read of somebody else's notes, or the store of record?** A copy
|
||||
needs conflict rules; a store of record needs export.
|
||||
14. **What happens to a note whose project, task or link target is deleted?**
|
||||
Answered here as "the link goes dead and the note is untouched". If your host
|
||||
app cascades instead, say so before the first delete.
|
||||
|
||||
---
|
||||
|
||||
## The shape of a note
|
||||
|
||||
```
|
||||
id an opaque string, unique
|
||||
project_id nullable — where it is filed, or loose
|
||||
title derived, never set by a caller
|
||||
body Markdown, the whole note
|
||||
created_by nullable — survives the account being deleted
|
||||
pinned favourite; the list puts these first
|
||||
archived_at nullable timestamp — put away, not thrown away
|
||||
deleted_at nullable timestamp — the trash
|
||||
created_at
|
||||
updated_at
|
||||
```
|
||||
|
||||
### Markdown in a column
|
||||
|
||||
Not rich text, and not a format of our own.
|
||||
|
||||
Export was a requirement before a line was written, and **if the stored form is
|
||||
the exported form there is no conversion left to lose anything in.** A note stays
|
||||
readable by any editor on the day the program is gone. It is also what makes
|
||||
search and anything an AI does over notes cheap: both read text.
|
||||
|
||||
The cost is real and you should know it before choosing: everything the editor can
|
||||
do has to survive a round trip through Markdown, and anything Markdown cannot say,
|
||||
the editor must not offer. There is no place to hide a feature.
|
||||
|
||||
### The title is the first line
|
||||
|
||||
Always, and re-derived on every save — not "when it is empty".
|
||||
|
||||
Derived once and then left alone, a title goes stale the moment somebody rewrites
|
||||
the opening of a note, and the list ends up calling a note by a name that is
|
||||
nowhere in it. **The column is a cache of the body, not a field beside it**, and no
|
||||
caller can set it, so every caller gets the same answer.
|
||||
|
||||
This forces one editor rule: a new note opens on its title, and the first line is
|
||||
styled as a heading. Otherwise a note whose first line is a sentence is a note
|
||||
named by its first sentence.
|
||||
|
||||
### Archive and trash are two different bits
|
||||
|
||||
The trash says *this was a mistake, and in thirty days it is gone*. The archive
|
||||
says *this is finished, and I do not want to read past it.*
|
||||
|
||||
A twelve-hundred-note import is exactly the case where most notes are worth
|
||||
keeping and almost none are worth looking at. Without an archive, the list is the
|
||||
whole archive forever, and finding this week's note means reading past a decade.
|
||||
|
||||
Both are nullable timestamps rather than booleans: *when was this put away* answers
|
||||
questions a flag cannot, and it costs the same byte.
|
||||
|
||||
---
|
||||
|
||||
## Filing: no hierarchy of its own
|
||||
|
||||
A note belongs to a project, or to nobody.
|
||||
|
||||
The reasoning is worth repeating because it is easy to talk yourself out of:
|
||||
projects already exist, are already shared, already have roles, already have a
|
||||
trash. **A second hierarchy to file things in is one too many**, and a note filed
|
||||
under a project inherits every one of those answers instead of needing its own.
|
||||
|
||||
- A note with a project: everybody who can read the project can read the note; an
|
||||
editor can change it.
|
||||
- A note without one: the author's, and nobody else's.
|
||||
|
||||
Sharing a note *is* filing it in a project. There is no access list of its own —
|
||||
a second way to give somebody access is a second place to get it wrong, and the
|
||||
two would disagree the day one of them changed.
|
||||
|
||||
---
|
||||
|
||||
## Links: three kinds, parsed out of the text
|
||||
|
||||
The body is the truth. Links are an index over it, rebuilt on every save.
|
||||
|
||||
| Syntax | Means | Notes |
|
||||
|---|---|---|
|
||||
| `#project` | a project | Same characters the task quick-add parser uses, on purpose |
|
||||
| `[[a note]]` | another note, by title | The spelling every note app has settled on |
|
||||
| `/opgave/<uuid>` or `task:<uuid>` | a task | What a link pasted out of the app carries |
|
||||
|
||||
Patterns as used here:
|
||||
|
||||
```
|
||||
project (?:^|\s)#([\p{L}\p{N}_-]{1,64})
|
||||
note \[\[([^\]\n]{1,200})\]\]
|
||||
task (?:/opgave/|task:)([0-9a-fA-F-]{36})
|
||||
```
|
||||
|
||||
Three decisions inside that table:
|
||||
|
||||
- **`#Firma` in a note means the same project as `#Firma` in the task box.** Not a
|
||||
second kind of tag that happens to look alike. That is the whole of "tag a note
|
||||
and see it in the project".
|
||||
- **The link table is torn down and rebuilt on save, not diffed.** A body is small
|
||||
and the set is small; a diff is a second description of the same fact, and it
|
||||
can disagree with the text. That is the one thing this must never do.
|
||||
- **The target is not a foreign key.** A link to something deleted is a fact about
|
||||
the note. It should read as a dead link rather than vanish, and it must not
|
||||
block the delete.
|
||||
|
||||
The index exists so the question can be asked **backwards**: *what has been written
|
||||
about this task?* Answering that by searching every note's prose would be too slow
|
||||
to leave a panel open on.
|
||||
|
||||
---
|
||||
|
||||
## Search
|
||||
|
||||
Three layers, in the order they answer.
|
||||
|
||||
### 1. Full-text, ranked
|
||||
|
||||
FTS5 over `title`, `body` and a folded copy, with `unicode61 remove_diacritics 2`.
|
||||
|
||||
Ranking is bm25 with **the title weighted ten times the body**. A word in the title
|
||||
is what the note is *about*; the same word once in a long body is a mention.
|
||||
|
||||
This matters more than it sounds. Ordering by "last touched" is invisible with ten
|
||||
notes and useless with twelve hundred imported the same evening: searching "Tesla"
|
||||
answered with a note about nginx first and the note actually called "Tesla Model Y"
|
||||
second — which reads as a search that does not work, because from where the person
|
||||
is sitting it is one.
|
||||
|
||||
### 2. Folding, for the language you are in
|
||||
|
||||
`remove_diacritics` does not touch `ø`, `æ` or `å` — Unicode considers them
|
||||
letters in their own right, not accented forms. So a Dane cannot find "grøn" by
|
||||
typing "gron", which is how people type when the keyboard is not theirs.
|
||||
|
||||
The answer here is a generated column carrying a transliterated copy:
|
||||
|
||||
```
|
||||
ø,Ø → o æ,Æ → ae å,Å → aa
|
||||
```
|
||||
|
||||
indexed alongside the body and **weighted the same as the body** — otherwise a
|
||||
folded match outranks an exact one.
|
||||
|
||||
> **Ask, do not copy.** This mapping is Danish. German wants `ü → ue`, Swedish
|
||||
> disagrees with Danish about `ä`, and Turkish has a dotless `ı` that breaks
|
||||
> naive lowercasing entirely. Get the mapping from somebody who types the
|
||||
> language.
|
||||
|
||||
### 3. A word inside a word
|
||||
|
||||
An index is built of tokens, and a token is whatever the tokenizer decided was a
|
||||
word. `gamerule keepInventory true` is three of them, and the middle one is
|
||||
`keepinventory` — so a search for "keep inventory" asks for two words that are
|
||||
nowhere and gets nothing, while the note sits right there.
|
||||
|
||||
A prefix query gets half of it: `keep*` matches, `inventory*` cannot, and the two
|
||||
are ANDed, so the half that works is thrown away by the half that cannot.
|
||||
|
||||
**This is not exotic in a notes app.** A decade of notes is full of commands,
|
||||
identifiers and file names, and every one of them is several words the index
|
||||
thinks is one.
|
||||
|
||||
So: the index answers first; when it has nothing, the text is read, and every term
|
||||
must appear somewhere in the note, in any word. A `LIKE '%term%'` over the folded
|
||||
column covers title, body and language folding at once.
|
||||
|
||||
What it costs: a leading `%` cannot use an index, so it scans. At twelve hundred
|
||||
notes that is a few megabytes and a millisecond or two, and it only runs when the
|
||||
fast path already came back empty — which is exactly the search that was about to
|
||||
disappoint somebody. **At a hundred thousand notes it is the wrong shape**, and the
|
||||
answer then is a trigram index, not a bigger scan.
|
||||
|
||||
It does not rank. There is no match to rank — that is the whole situation.
|
||||
|
||||
---
|
||||
|
||||
## The list, and why the body is empty in it
|
||||
|
||||
A list showing a title and ninety characters of preview must not fetch whole
|
||||
notes. Twelve hundred notes came to 3.9 MB, of which 98 % was body nobody could
|
||||
see.
|
||||
|
||||
The important half is *how* it is trimmed: **the body is emptied, not shortened.**
|
||||
|
||||
A field called `body` holding a *piece* of the body is a trap. The editor opens
|
||||
it, the autosave pause writes it back, and the note is now cut down to its own
|
||||
excerpt — destroyed by nothing more than having been looked at. Send `preview` as
|
||||
its own field and leave `body` empty; the client sees the empty body and fetches
|
||||
the whole note.
|
||||
|
||||
That costs one more separation, and it is not optional:
|
||||
|
||||
> **Which row is selected and which note is loaded are two different things.**
|
||||
|
||||
The editor loads its text when the note's id changes. Hand it the excerpt first
|
||||
and the whole note second and the id is the same both times — so the second one
|
||||
never renders, and somebody sits looking at a note that had a title a moment ago.
|
||||
|
||||
Highlight the row immediately from the list; give the editor the note only when it
|
||||
is whole.
|
||||
|
||||
Rendering cost, separately: with twelve hundred rows, first paint took 2.1 s. CSS
|
||||
`content-visibility: auto` with an intrinsic-size guess took it to 0.95 s with no
|
||||
virtual list, no scroll maths, and no list that can fall out of step with itself.
|
||||
Try that before building windowing.
|
||||
|
||||
---
|
||||
|
||||
## The editor contract
|
||||
|
||||
Rich text on top, Markdown underneath, and a source view that shows the file.
|
||||
|
||||
The supported subset — deliberately small, because everything in it has to survive
|
||||
the round trip:
|
||||
|
||||
| Block | Markdown |
|
||||
|---|---|
|
||||
| Title | `# ` |
|
||||
| Heading | `## ` |
|
||||
| Subheading | `### ` |
|
||||
| Quote | `> ` |
|
||||
| Code block | ```` ``` ```` with an optional language |
|
||||
| Bullet and numbered lists | `- `, `1. `, nested |
|
||||
|
||||
Inline: `**bold**`, `*italic*`, `~~strikethrough~~`, `` `code` ``, `<u>underline</u>`,
|
||||
and images as ``.
|
||||
|
||||
Five rules that were each learned the hard way:
|
||||
|
||||
1. **Every line becomes exactly one block.** Markdown normally joins consecutive
|
||||
lines into a paragraph. Here it must not: the person typed a line and expects
|
||||
to get it back, and an editor that silently reflows what somebody wrote is an
|
||||
editor they stop trusting.
|
||||
2. **Load on id change, never while typing.** Writing to the editor's HTML puts
|
||||
the caret back at the start, which mid-sentence is unusable.
|
||||
3. **One definition of what a code block contains.** A copy button inside the
|
||||
`<pre>` and a syntax highlighter reading `textContent` will disagree, and the
|
||||
difference ends up in the file — a click on "Copy" wrote the word *Copied* into
|
||||
the code and the next save put it on disk. Have one function that says what a
|
||||
block holds, used by the highlighter and the Markdown serialiser both.
|
||||
4. **A note that cannot be rendered must not take the pane with it.** One note
|
||||
arriving in an unexpected shape threw inside the editor's reactive effect, and
|
||||
from then on *no* note would open — the previous one just stayed on screen. Put
|
||||
a guard around the render, fall back to showing the raw text, and let the rest
|
||||
of the pane keep working.
|
||||
5. **Escaping between tags and escaping inside an attribute are different jobs.**
|
||||
An escaper written for text between two tags handles `&`, `<`, `>` — and not
|
||||
quotes, because between tags they are harmless. Inside an attribute they close
|
||||
it. `` is the whole exploit.
|
||||
|
||||
---
|
||||
|
||||
## Attachments
|
||||
|
||||
Images pasted or dragged into a note, stored content-addressed by hash, referenced
|
||||
from the body as ordinary Markdown images.
|
||||
|
||||
The security shape, which is the part worth copying:
|
||||
|
||||
- **Served as `application/octet-stream` with `nosniff` by default.** Attachments
|
||||
are the one place a user supplies content another user opens. An uploaded SVG or
|
||||
HTML file rendered inline runs its own script on your origin with the session
|
||||
cookie attached.
|
||||
- **One exception, as a named allowlist:** a short list of raster image types
|
||||
served inline as their exact type — `png`, `jpeg`, `gif`, `webp`, `avif`, `heic`.
|
||||
A note carries its pictures in the text, and served as octet-stream every one of
|
||||
them is a broken image.
|
||||
- **The list is named one type at a time, never matched on an `image/` prefix.**
|
||||
The prefix lets `image/svg+xml` through, and an SVG is a document that can run
|
||||
script.
|
||||
- **The stored path is checked against the files directory again at read time.** A
|
||||
traversal that somehow got into the database must not become one that reads
|
||||
`/etc/passwd`.
|
||||
- **Reaching an attachment follows the note, which follows its project.** Same
|
||||
chain as everything else. Do not let a client hand you the parent id and take it
|
||||
on trust.
|
||||
|
||||
---
|
||||
|
||||
## Import and export
|
||||
|
||||
The same format both ways: **a zip of Markdown files**, one per note, with a YAML
|
||||
front matter block.
|
||||
|
||||
```markdown
|
||||
---
|
||||
created: 2019-04-02T10:15:00Z
|
||||
modified: 2024-11-30T21:04:00Z
|
||||
---
|
||||
|
||||
# Møde med Anders
|
||||
|
||||
Han vil gerne have kaffe hver uge.
|
||||
```
|
||||
|
||||
- `created` / `created_at` / `date` and `modified` / `updated` / `updated_at` are
|
||||
all accepted on the way in. Obsidian and Bear write this block; so should you.
|
||||
- Images travel in the zip beside the notes, and the links in the text are
|
||||
relative — **a note and its pictures must end up in the same archive**, which
|
||||
matters the moment an export is split into parts.
|
||||
- File names collide: two notes called "Møde" are two notes, and a zip with one
|
||||
entry twice is a zip that loses one.
|
||||
|
||||
Limits that a real import taught, all of them cheap to add and expensive to omit:
|
||||
|
||||
- A ceiling on the upload, on the number of entries, and on the **unpacked** size.
|
||||
64 MB of zip expanded to 66 GB is a normal shape for an attack.
|
||||
- A per-file limit as well as a whole-archive budget: one enormous photograph and
|
||||
ten thousand small ones are different attacks with the same ending.
|
||||
- The entry name is never used as a path. Store by hash; the name is only a key
|
||||
for rewriting links.
|
||||
- **Count what you skip, and say so.** A note whose name contained an ellipsis was
|
||||
dropped silently by a filter looking for `..` — one out of 1141, found only by
|
||||
counting at both ends.
|
||||
|
||||
### Timestamps on the way in
|
||||
|
||||
An importer needs to write `created_at` and `updated_at` from the file, and the
|
||||
normal save path must refuse to let it.
|
||||
|
||||
`updated_at` means *when did this program last write this note*. Let any caller set
|
||||
it and the column means whatever the last writer felt like. Give the importer its
|
||||
own narrow entry point instead, and say in the comment why it exists.
|
||||
|
||||
Without those dates, a move of twelve hundred notes is a pile that was all written
|
||||
the same evening — and the order they were written in is half of what an archive
|
||||
is.
|
||||
|
||||
---
|
||||
|
||||
## What is still open here
|
||||
|
||||
Written down because a specification that only lists what works is a specification
|
||||
that lies by omission.
|
||||
|
||||
- **Note bodies are plaintext at rest.** They hold API keys and passwords, and
|
||||
they are in every backup anybody can download. Encrypting them is bigger than it
|
||||
sounds: the body is in the full-text index and the folded column is generated
|
||||
from it, so both are a second plaintext copy in the same database. Encrypt and
|
||||
both have to go, and search has to be rewritten to decrypt in application code —
|
||||
fine at a thousand notes, a full scan per keystroke at ten thousand, at which
|
||||
point the answer is a keyed index (an HMAC per word).
|
||||
- **Import never merges.** Every run creates new notes. Say so out loud in the UI.
|
||||
- **There is no sync.** Export and the API are the way out.
|
||||
|
||||
---
|
||||
|
||||
## A short list of the traps, for the person in a hurry
|
||||
|
||||
1. A `body` field holding an excerpt will be saved over the real one.
|
||||
2. Selected row and loaded note are two different pieces of state.
|
||||
3. The title must be re-derived on every save or it goes stale.
|
||||
4. Diacritic stripping is not folding, and folding is language-specific.
|
||||
5. A tokenizer cannot see a word inside a word; decide what you do about it.
|
||||
6. An attachment served inline is a script on your origin unless you allowlist by
|
||||
exact type and send `nosniff`.
|
||||
7. Escaping for text and escaping for attributes are different functions.
|
||||
8. One render failure must not be allowed to freeze the whole pane.
|
||||
9. An unpacked-size limit is not optional on an archive import.
|
||||
10. Let the importer set timestamps; let nobody else.
|
||||
@@ -6,7 +6,7 @@
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<img src="site/screenshots/today.png" alt="The Today view: projects, filters and labels in the sidebar, beside the tasks due today." width="900">
|
||||
<img src="docs/screenshots/today.png" alt="The Today view: projects, filters and labels in the sidebar, beside the tasks due today." width="900">
|
||||
</p>
|
||||
|
||||
---
|
||||
@@ -35,6 +35,11 @@ mail-to-task address, and an MCP endpoint so Claude can work your task list dire
|
||||
in Firma, labelled regnskab. Danish and English, mixed freely in one line.
|
||||
- **Projects, sections, sub-tasks, labels, saved filters.** List, board and calendar
|
||||
views, and foldable groups over the projects in the sidebar.
|
||||
- **Notes, beside the work.** Rich text that is Markdown underneath — headings,
|
||||
lists, quotes, fenced code. A note is shared by filing it in a project;
|
||||
`#project` and `[[another note]]` are links, and a task shows the notes that
|
||||
mention it. A zip of Markdown goes in — Obsidian, Bear, or Apple Notes through
|
||||
the exporter in [`tools/`](tools/) — and comes back out the same shape.
|
||||
- **Drag where it means something.** Reorder a list or a board, file a project under
|
||||
a group, drop a task on another project, or drop it on another day.
|
||||
- **Sharing.** Owner, editor and viewer roles. There is no open registration:
|
||||
@@ -90,21 +95,24 @@ Apple Reminders, forward mail into it, and add it to Claude as a connector.
|
||||
| Project templates | Done, tested |
|
||||
| Nightly backups with rotation | Done, tested |
|
||||
| Comments, attachments, notifications, Web Push | Built |
|
||||
| Notes: rich text, Markdown on disk, links to projects, notes and tasks | Done, tested |
|
||||
| Notes in and out: a zip of Markdown, both directions | Done, tested |
|
||||
| Todoist CSV import and export, account export | Done, tested |
|
||||
| MCP server for Claude | Done, tested |
|
||||
| CalDAV server (two-way VTODO) | Done, tested |
|
||||
| Mail-to-task | Done, tested |
|
||||
| AI layer (Anthropic/OpenAI/Google/local) | Built, degrades when unconfigured |
|
||||
| Gmail: OAuth2 with PKCE, polling, one-way to tasks | Done, tested |
|
||||
| Google Calendar: read-only, laid over the task calendar | Built, never run against Google |
|
||||
| Update notice for administrators | Done, tested |
|
||||
| Personal API tokens | Done, tested |
|
||||
| OpenAPI 3.1 spec, checked against the router | Done, tested |
|
||||
| End-to-end smoke tests (Playwright) | Done |
|
||||
| Documentation, landing page, licence | Done |
|
||||
|
||||
**Not finished:** Web Push, the AI providers and the Gmail API calls are
|
||||
implemented against their specifications and have only ever been exercised there,
|
||||
not against live services. Everything else has an interface: opening a task shows
|
||||
**Not finished:** Web Push, the AI providers, and the Gmail and Google Calendar
|
||||
API calls are implemented against their specifications and have only ever been
|
||||
exercised there, not against live services. Everything else has an interface: opening a task shows
|
||||
its description, sub-tasks, comments, files and reminders, and the settings pages
|
||||
cover the account, notifications, integrations, AI, API tokens, import, export and
|
||||
templates.
|
||||
@@ -116,17 +124,20 @@ accented `o` and a Danish-first app that cannot do this is broken for its own us
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
|  |  |
|
||||
|  |  |
|
||||
| Board view — one column per section. | The light theme. |
|
||||
|  |  |
|
||||
| A week at a time, so a task can be dragged across a month boundary. | A task, opened. |
|
||||
|  |  |
|
||||
| Five themes, all measured for contrast. | What has been done on the server. |
|
||||
|  |  |
|
||||
| A week at a time, so a task can be dragged across a month boundary. | A task, opened: description, priority, labels, sub-tasks and its files. |
|
||||
|  |  |
|
||||
| Notes, filed in a project to share them. | The same note as its source — which is what the export writes to disk. |
|
||||
|  |  |
|
||||
| Five themes and four looks: how bright, and how it reads. | What has been done on the server. |
|
||||
|
||||
Regenerate these with `node scripts/generate-screenshots.js` from `web/`. It starts
|
||||
the real binary, creates its own account, fills it with demo data and photographs
|
||||
eleven surfaces — nothing to set up and nothing to remember. (`go run ./tools/shots`
|
||||
still works, and is the way to photograph an instance that is already running.)
|
||||
the real binary, creates its own account and a second one to share with, fills them
|
||||
with demo data and photographs fourteen surfaces — nothing to set up and nothing to
|
||||
remember. (`go run ./tools/shots` still works, and is the way to photograph an
|
||||
instance that is already running.)
|
||||
|
||||
## Running it
|
||||
|
||||
|
||||
@@ -10,13 +10,28 @@ reply within a week.
|
||||
|
||||
- **Passwords** are Argon2id with the parameters recorded in the hash, so they can
|
||||
be raised later without locking anyone out.
|
||||
- **Every token** — session, invite, reset, API, calendar feed, mail address — is
|
||||
stored only as a SHA-256 hash. A database dump contains nothing that can be
|
||||
presented.
|
||||
- **Session, invite, reset and API tokens** are stored only as a SHA-256 hash. A
|
||||
database dump contains nothing that can be presented.
|
||||
- **The calendar feed address and the mail-to-task address are not hashed.** They
|
||||
are looked up by their exact value, and they are shown in the interface more
|
||||
than once, which a hash cannot do. Anybody holding a backup file therefore holds
|
||||
those two URLs: the feed reads that person's tasks without a login, and the mail
|
||||
address files tasks into their inbox. Rotating either from **Settings →
|
||||
Integrations** invalidates the old one. This is written down rather than fixed
|
||||
because fixing it means the address can only ever be shown once, which is a
|
||||
decision about how the feature works and not only about how it is stored.
|
||||
- **Mailbox passwords, OAuth tokens, TOTP secrets and AI keys** are sealed with
|
||||
AES-GCM under a key kept outside the database — see `internal/secret`. That is
|
||||
what makes a stray copy of the database less than a stray copy of everything:
|
||||
the backup does not carry the key.
|
||||
- **Sessions** are `httpOnly`, `SameSite=Lax`, and `__Host-` prefixed over HTTPS.
|
||||
- **Uploads** are stored content-addressed by hash and always served as
|
||||
`application/octet-stream` with `nosniff`, so an uploaded SVG cannot run script
|
||||
on this origin.
|
||||
- **Uploads** are stored content-addressed by hash and served as
|
||||
`application/octet-stream` with `nosniff` — with one exception, added so that a
|
||||
note can show the pictures in it: a short allowlist of raster image types is
|
||||
served inline as its own type. The list is named one format at a time rather
|
||||
than matched on an `image/` prefix, because the prefix would let `image/svg+xml`
|
||||
through, and an SVG is a document that can run script. `nosniff` stays, so the
|
||||
browser must read the file as that image or not at all.
|
||||
- **The filter language** compiles to parameterised SQL. No typed text is ever
|
||||
concatenated into a statement.
|
||||
- **Permission checks** are route middleware, and anything you may not see answers
|
||||
|
||||
@@ -75,7 +75,9 @@ func run() error {
|
||||
runner := jobs.New(cfg, db, log, api.Mail(), api.Hub())
|
||||
runner.SyncGmail = api.SyncGmail
|
||||
runner.SyncMailbox = api.SyncMailbox
|
||||
runner.SyncCalendars = api.SyncCalendars
|
||||
runner.Push = api.PushToUser
|
||||
runner.SendBeacon = api.SendBeacon
|
||||
runner.Start(jobCtx)
|
||||
|
||||
srv := &http.Server{
|
||||
|
||||
@@ -1,13 +1,14 @@
|
||||
# Calendar and CalDAV
|
||||
|
||||
Two ways to get your tasks into a calendar, and they are not the same thing.
|
||||
Two ways to get your tasks into a calendar, one way to get a calendar into
|
||||
verdande, and none of the three is the same thing.
|
||||
|
||||
| | [Feed](#subscribing-to-a-feed) | [CalDAV](#connecting-a-caldav-client) |
|
||||
|---|---|---|
|
||||
| Direction | Read only | Two-way |
|
||||
| Works with | Anything that subscribes to a URL | Apple Reminders, Thunderbird, DAVx⁵ |
|
||||
| You can tick things off | No | Yes |
|
||||
| Setup | Paste a URL | Username and token |
|
||||
| | [Feed](#subscribing-to-a-feed) | [CalDAV](#connecting-a-caldav-client) | [Google Calendar](#showing-a-google-calendar-in-verdande) |
|
||||
|---|---|---|---|
|
||||
| Direction | Tasks out, read only | Tasks out, two-way | Events **in**, read only |
|
||||
| Works with | Anything that subscribes to a URL | Apple Reminders, Thunderbird, DAVx⁵ | A Google account |
|
||||
| You can tick things off | No | Yes | No — an event is not a task |
|
||||
| Setup | Paste a URL | Username and token | Connect, then pick calendars |
|
||||
|
||||
## Subscribing to a feed
|
||||
|
||||
@@ -39,6 +40,42 @@ most people actually keep.
|
||||
**Settings → Calendar feed → Rotate** issues a new one and breaks the old
|
||||
immediately.
|
||||
|
||||
## Showing a Google calendar in verdande
|
||||
|
||||
The two sections above send your tasks *out*. This is the other direction: your
|
||||
Google calendar drawn underneath them, so one screen answers "what does this day
|
||||
already have in it".
|
||||
|
||||
**Settings → Integrations → Google Calendar → Connect**, then tick the calendars
|
||||
you want and press Save. They appear in **Calendar** in the sidebar, in Google's
|
||||
own colour for each one, beside your own tasks with due dates.
|
||||
|
||||
It needs the same OAuth client Gmail uses, with the Calendar API enabled and the
|
||||
second redirect URI registered — see [Configuration](configuration.md#google).
|
||||
|
||||
!!! warning "Read only"
|
||||
verdande never writes to your Google calendar. An event cannot be dragged,
|
||||
edited or ticked off here; clicking one opens it in Google, which is where it
|
||||
can be changed. Your own tasks in the same grid still drag to another day as
|
||||
they always have.
|
||||
|
||||
Writing would need the `calendar.events` scope and an answer to what should
|
||||
happen when both sides moved the same meeting — which is a synchronisation
|
||||
model rather than a scope, and it is deliberately not built yet.
|
||||
|
||||
!!! note "The Internal client applies here too"
|
||||
An OAuth client registered as *Internal* in a Google Workspace can only be
|
||||
used by accounts in that workspace, calendars included. A personal
|
||||
`@gmail.com` address is refused with `org_internal`. For one personal account
|
||||
the shorter road is the calendar's own secret address: in Google Calendar,
|
||||
**Settings → the calendar → Integrate calendar → Secret address in iCal
|
||||
format**. Nothing in verdande subscribes to one yet — it is the next piece of
|
||||
this feature.
|
||||
|
||||
Events are kept for ninety days back and a year forward, refreshed every fifteen
|
||||
minutes and whenever you press **Fetch now**. Paging the grid past that window
|
||||
says so rather than showing an empty month.
|
||||
|
||||
## Connecting a CalDAV client
|
||||
|
||||
This is the one where ticking something off in Apple Reminders ticks it off here.
|
||||
|
||||
@@ -39,12 +39,24 @@ Any Go duration: `720h`, `30m`, `45s`.
|
||||
yourself, password resets are written to the log, and reminders arrive in the
|
||||
app and as Web Push rather than by email.
|
||||
|
||||
## Gmail
|
||||
## Google
|
||||
|
||||
Only needed if anyone will connect a mailbox. Register an OAuth client in Google
|
||||
Cloud (**APIs & Services → Credentials → OAuth client ID → Web application**) with
|
||||
the redirect URI `https://your-address/oauth/gmail/callback`, enable the Gmail API,
|
||||
and add the `gmail.readonly` scope.
|
||||
Only needed if anyone will connect a Gmail mailbox or a Google calendar. One
|
||||
registration covers both: register an OAuth client in Google Cloud
|
||||
(**APIs & Services → Credentials → OAuth client ID → Web application**) and add
|
||||
**both** redirect URIs:
|
||||
|
||||
```
|
||||
https://your-address/oauth/gmail/callback
|
||||
https://your-address/oauth/calendar/callback
|
||||
```
|
||||
|
||||
Then enable the APIs and scopes for whichever you want:
|
||||
|
||||
| For | Enable | Scope |
|
||||
|---|---|---|
|
||||
| Mailboxes | Gmail API | `gmail.readonly` |
|
||||
| Calendar | Google Calendar API | `calendar.readonly` |
|
||||
|
||||
| Variable | What it does |
|
||||
|---|---|
|
||||
@@ -52,8 +64,15 @@ and add the `gmail.readonly` scope.
|
||||
| `VERDANDE_GMAIL_CLIENT_SECRET` | The client secret. |
|
||||
|
||||
One registration serves every user on the instance; each person authorises their
|
||||
own mailbox from **Settings → Gmail**. The redirect URI is derived from
|
||||
`VERDANDE_BASE_URL`, so it cannot drift out of step with what you registered.
|
||||
own mailbox from **Settings → Integrations → Gmail** and their own calendar from
|
||||
**Settings → Integrations → Google Calendar**. Both redirect URIs are derived from
|
||||
`VERDANDE_BASE_URL`, so they cannot drift out of step with what you registered.
|
||||
|
||||
!!! note "Two connections, two grants"
|
||||
Connecting a calendar does not disturb a working Gmail connection, and
|
||||
disconnecting one does not take the other with it. Google issues a refresh
|
||||
token per authorisation and keeps up to a hundred of them live per account,
|
||||
each carrying only the scopes it was granted with.
|
||||
|
||||
## Secrets
|
||||
|
||||
|
||||
@@ -22,6 +22,9 @@ forward mail to, and an MCP endpoint so Claude can work your task list directly.
|
||||
Firma, labelled regnskab. Danish and English, mixed freely in one line.
|
||||
- **Projects, sections, sub-tasks, labels, [saved filters](filters.md).** List,
|
||||
board and calendar views, and foldable groups over the projects in the sidebar.
|
||||
- **Notes, beside the work.** Rich text to write in, Markdown on disk. A note is
|
||||
shared by filing it in a project; `#project` and `[[another note]]` are links,
|
||||
and a task shows what has been written about it.
|
||||
- **Drag where it means something** — reorder a list or a board, file a project
|
||||
under a group, drop a task on another project, or drop it on another day.
|
||||
- **Sharing** with owner, editor and viewer roles. Anyone outside your instance
|
||||
@@ -31,6 +34,33 @@ forward mail to, and an MCP endpoint so Claude can work your task list directly.
|
||||
- **[Calendar](caldav.md), [mail](mail.md) and [Claude](mcp.md)** connected over
|
||||
open standards rather than private formats.
|
||||
|
||||
## What it looks like
|
||||
|
||||

|
||||
|
||||
*Upcoming, as a month. A week and a plain list are the other two, and a task can be
|
||||
dragged from one day onto another.*
|
||||
|
||||

|
||||
|
||||
*The same project as a list, where sections read as bands. As a board they are
|
||||
columns.*
|
||||
|
||||

|
||||
|
||||
*What you have handed to somebody else, grouped by whoever has it.*
|
||||
|
||||

|
||||
|
||||
*Where [Gmail](mailboxes.md), the [calendar feed](caldav.md) and the
|
||||
[mail-to-task address](mail.md) are connected.*
|
||||
|
||||

|
||||
|
||||
*Inviting somebody to the instance. There is no open registration — everybody
|
||||
arrives through a link, and picks their own password. See
|
||||
[signing in](signing-in.md).*
|
||||
|
||||
## Getting it running
|
||||
|
||||
verdande is built to be deployed as a **Rune** in
|
||||
|
||||
@@ -16,7 +16,7 @@ administrator who can read yours.
|
||||
**Settings → Integrations → Gmail → Connect.**
|
||||
|
||||
Gmail needs an OAuth client, which is the instance's registration with Google
|
||||
rather than anybody's account — see [Configuration](configuration.md#gmail).
|
||||
rather than anybody's account — see [Configuration](configuration.md#google).
|
||||
Once it is set up, each person connects their own mailbox through it.
|
||||
|
||||
Choose what makes a task:
|
||||
|
||||
@@ -263,6 +263,36 @@ paths:
|
||||
"204": { description: Saved. }
|
||||
"400": { $ref: "#/components/responses/BadRequest" }
|
||||
|
||||
/api/v1/auth/nav-order:
|
||||
put:
|
||||
tags: [auth]
|
||||
summary: The order of the fixed views in the sidebar.
|
||||
description: |
|
||||
I dag, Kommende, Venter and the inbox, in whatever order this person wants
|
||||
them. On the account rather than in the browser: which order you want them
|
||||
in is a fact about you and should be the same on the phone as on the laptop.
|
||||
|
||||
A list of keys rather than a number per view. The set of views changes with
|
||||
the program, and a key that no longer exists costs nothing — the sidebar
|
||||
draws what it recognises and appends anything new at the end, which is what
|
||||
makes adding a view later not a migration. An empty list means the order the
|
||||
program ships with.
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
order:
|
||||
type: array
|
||||
maxItems: 20
|
||||
items: { type: string }
|
||||
example: ["inbox", "today", "upcoming", "delegated"]
|
||||
responses:
|
||||
"204": { description: Saved. }
|
||||
"400": { $ref: "#/components/responses/BadRequest" }
|
||||
|
||||
/api/v1/auth/me:
|
||||
get:
|
||||
tags: [auth]
|
||||
@@ -1366,6 +1396,204 @@ paths:
|
||||
description: The environment variables that are not set.
|
||||
"403": { $ref: "#/components/responses/Forbidden" }
|
||||
|
||||
/api/v1/notes/bulk:
|
||||
post:
|
||||
tags: [notes]
|
||||
summary: Archive, unarchive or delete a set of notes in one call.
|
||||
description: |
|
||||
One call rather than one per note, because the thing being asked is one
|
||||
thing: "put these away". Fifty separate requests would be fifty round
|
||||
trips, fifty chances for half of them to fail, and a list that redraws
|
||||
fifty times — and after an import of twelve hundred notes, selecting fifty
|
||||
is the ordinary case rather than the extreme one.
|
||||
|
||||
Each note is still checked on its own. A set is a convenience for the
|
||||
caller and never a way to touch something they could not have touched one
|
||||
at a time; anything they may not write to is counted in `skipped` rather
|
||||
than failing the whole call, so one note somebody else just deleted does
|
||||
not stop the other forty-nine.
|
||||
|
||||
Archiving is not deleting. The trash says "this was a mistake, and in
|
||||
thirty days it is gone"; the archive says "this is finished, and I want to
|
||||
keep it".
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
required: [ids]
|
||||
properties:
|
||||
ids:
|
||||
type: array
|
||||
items: { type: string }
|
||||
description: At most 500.
|
||||
archived:
|
||||
type: boolean
|
||||
description: Put away, or bring back out.
|
||||
delete:
|
||||
type: boolean
|
||||
description: Into the trash instead. Wins over `archived`.
|
||||
responses:
|
||||
"200":
|
||||
description: How many were changed, and how many were passed over.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
done: { type: integer }
|
||||
skipped: { type: integer }
|
||||
"422":
|
||||
description: No ids, too many, or neither archived nor delete was asked for.
|
||||
|
||||
/api/v1/notes/{noteID}/attachments:
|
||||
post:
|
||||
tags: [notes]
|
||||
summary: Put a file into a note.
|
||||
description: |
|
||||
A picture pasted or dropped into a note. Until this existed the import
|
||||
could bring files in and nothing else could, so the most ordinary act in
|
||||
a notes app — paste a screenshot — did nothing at all.
|
||||
|
||||
Editor rights on the note. The same answer for "no such note" and "not
|
||||
yours", so an id cannot be probed.
|
||||
|
||||
The response carries the attachment's URL; the editor writes that into the
|
||||
note's text as a Markdown image. The file is never inlined as a data URL —
|
||||
a photograph as base64 would make the note itself megabytes of text.
|
||||
parameters:
|
||||
- name: noteID
|
||||
in: path
|
||||
required: true
|
||||
schema: { type: string }
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
multipart/form-data:
|
||||
schema:
|
||||
type: object
|
||||
required: [file]
|
||||
properties:
|
||||
file:
|
||||
type: string
|
||||
format: binary
|
||||
responses:
|
||||
"201":
|
||||
description: Stored, and hung on the note.
|
||||
"404":
|
||||
description: No such note, or not yours to write in.
|
||||
"413":
|
||||
description: Larger than 25 MB.
|
||||
|
||||
/api/v1/beacon:
|
||||
post:
|
||||
tags: [auth]
|
||||
summary: Report that an installation exists. Unauthenticated.
|
||||
description: |
|
||||
The receiving end of the beacon. Open on purpose: the caller is a
|
||||
stranger's installation, which is the whole feature.
|
||||
|
||||
Answers 404 unless this instance has been told it is the collector, so an
|
||||
ordinary installation is not a spam sink that answers 200 to anything.
|
||||
Rate limited like the login routes, and the table behind it has a ceiling:
|
||||
the endpoint inserts a row keyed on an id the caller invented, so without
|
||||
one anybody could grow it until the volume is full.
|
||||
|
||||
Nothing about the request is recorded but the two values in the body. No
|
||||
source address is read or written here.
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
required: [instance_id]
|
||||
properties:
|
||||
instance_id:
|
||||
type: string
|
||||
description: A random id the installation generated for itself.
|
||||
version:
|
||||
type: string
|
||||
responses:
|
||||
"204":
|
||||
description: Recorded.
|
||||
"400":
|
||||
description: The id or the version is not the expected shape.
|
||||
"404":
|
||||
description: This instance is not a collector.
|
||||
|
||||
/api/v1/beacon/count:
|
||||
get:
|
||||
tags: [auth]
|
||||
summary: How many installations reported in the last thirty days.
|
||||
description: |
|
||||
Unauthenticated and cached for a minute, and only answered when the
|
||||
collector has been told to publish it.
|
||||
|
||||
A window rather than a running total: a total only climbs, so it stops
|
||||
meaning "how many people use this" and starts meaning "how many ever
|
||||
tried it".
|
||||
responses:
|
||||
"200":
|
||||
description: The count.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
installs: { type: integer }
|
||||
window_days: { type: integer }
|
||||
"404":
|
||||
description: Not a collector, or not published.
|
||||
|
||||
/api/v1/beacon/settings:
|
||||
get:
|
||||
tags: [auth]
|
||||
summary: What this instance reports, and to whom. Administrators only.
|
||||
description: |
|
||||
Includes the exact payload that will be sent, so the settings page can
|
||||
print it in full rather than describe it.
|
||||
responses:
|
||||
"200":
|
||||
description: The current configuration, and the counts if this is the collector.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
enabled: { type: boolean }
|
||||
collector_url: { type: string }
|
||||
instance_id: { type: string }
|
||||
version: { type: string }
|
||||
is_collector: { type: boolean }
|
||||
publish_count: { type: boolean }
|
||||
last_ping_at: { type: string, format: date-time }
|
||||
put:
|
||||
tags: [auth]
|
||||
summary: Turn the beacon off, or point it somewhere else. Administrators only.
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
enabled: { type: boolean }
|
||||
collector_url:
|
||||
type: string
|
||||
description: |
|
||||
Where reports go. Fetched through a dialer that refuses
|
||||
loopback and private address space, so this cannot be used to
|
||||
make the server reach inside its own network.
|
||||
is_collector: { type: boolean }
|
||||
publish_count: { type: boolean }
|
||||
responses:
|
||||
"200":
|
||||
description: The configuration as it now stands.
|
||||
"422":
|
||||
description: The collector address is not an http or https URL.
|
||||
|
||||
/api/v1/panel/restart:
|
||||
post:
|
||||
tags: [auth]
|
||||
@@ -1821,6 +2049,125 @@ paths:
|
||||
"404": { $ref: "#/components/responses/NotFound" }
|
||||
"422": { $ref: "#/components/responses/Validation" }
|
||||
|
||||
/api/v1/projects/{projectID}/sections/reorder:
|
||||
parameters:
|
||||
- { name: projectID, in: path, required: true, schema: { type: string } }
|
||||
post:
|
||||
tags: [projects]
|
||||
summary: Set the order the project's sections appear in. Editors and owners.
|
||||
description: |
|
||||
The whole list at once, for the same reasons as `/projects/reorder`: a
|
||||
project has a handful of sections, so sending the order you want is
|
||||
simpler than midpoints and cannot land half-applied.
|
||||
|
||||
Ids belonging to another project are skipped rather than refused.
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
required: [ids]
|
||||
properties:
|
||||
ids:
|
||||
type: array
|
||||
maxItems: 500
|
||||
items: { type: string }
|
||||
description: Section ids, top to bottom.
|
||||
responses:
|
||||
"204": { description: Reordered. }
|
||||
"404": { $ref: "#/components/responses/NotFound" }
|
||||
"422": { $ref: "#/components/responses/Validation" }
|
||||
|
||||
/api/v1/calendar/subscriptions:
|
||||
post:
|
||||
tags: [calendar]
|
||||
summary: Subscribe to a calendar you only have an address to.
|
||||
description: |
|
||||
An ICS or `webcal://` address. Several per person, unlike the Google
|
||||
connection, which is one — the address is the identity here, the way the
|
||||
token is for OAuth.
|
||||
|
||||
The address is fetched once before it is stored: an address that cannot be
|
||||
read is a typo far more often than a server that happens to be down, and
|
||||
saying so while somebody is still looking at the field is worth two
|
||||
seconds. `webcal://` is rewritten to `https://`, and the destination is
|
||||
checked against the private-network wall both when it is written down and
|
||||
again on the resolved address every time it is fetched.
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
required: [url]
|
||||
properties:
|
||||
url: { type: string, description: "https:// or webcal://" }
|
||||
name:
|
||||
type: string
|
||||
description: Optional. The calendar's own name is used when this is empty.
|
||||
responses:
|
||||
"201":
|
||||
description: Subscribed, and fetched once.
|
||||
"422": { $ref: "#/components/responses/Validation" }
|
||||
|
||||
/api/v1/calendar/subscriptions/{id}:
|
||||
parameters:
|
||||
- { name: id, in: path, required: true, schema: { type: string } }
|
||||
delete:
|
||||
tags: [calendar]
|
||||
summary: Stop subscribing. The cached events go with it.
|
||||
responses:
|
||||
"204": { description: Gone. }
|
||||
"404": { $ref: "#/components/responses/NotFound" }
|
||||
|
||||
/api/v1/push/test:
|
||||
post:
|
||||
tags: [notifications]
|
||||
summary: Send one notification to your own devices, and say what happened.
|
||||
description: |
|
||||
The answer to "no notifications arrive". A push that a service refuses is
|
||||
the ordinary failure — a stale subscription, a browser that revoked
|
||||
permission without telling the server — and every one of them looks
|
||||
exactly like silence from the outside.
|
||||
|
||||
It goes through the same path a real notification takes, so what it proves
|
||||
is the thing that has to work. The title and body come from the caller
|
||||
because the dictionaries live in the interface.
|
||||
requestBody:
|
||||
required: false
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
title: { type: string, maxLength: 200 }
|
||||
body: { type: string, maxLength: 500 }
|
||||
responses:
|
||||
"200":
|
||||
description: What each device did with it.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
subscriptions:
|
||||
type: integer
|
||||
description: Devices registered for this account.
|
||||
sent:
|
||||
type: integer
|
||||
failed:
|
||||
type: array
|
||||
items:
|
||||
type: object
|
||||
properties:
|
||||
service:
|
||||
type: string
|
||||
description: The push service's host — never the endpoint, which is a credential.
|
||||
reason:
|
||||
type: string
|
||||
description: What the service said.
|
||||
|
||||
/api/v1/sections/{sectionID}:
|
||||
parameters:
|
||||
- { name: sectionID, in: path, required: true, schema: { type: string } }
|
||||
@@ -2477,6 +2824,283 @@ paths:
|
||||
created: { type: integer }
|
||||
"502": { $ref: "#/components/responses/Upstream" }
|
||||
|
||||
/api/v1/calendar:
|
||||
get:
|
||||
tags: [integrations]
|
||||
summary: The Google Calendar connection and the calendars in it.
|
||||
description: |
|
||||
Read-only. verdande shows a Google calendar beside its own dated tasks and
|
||||
never writes one back — writing needs the `calendar.events` scope and an
|
||||
answer to what happens when both sides moved the same meeting, which is a
|
||||
synchronisation model rather than a scope.
|
||||
|
||||
The list of calendars is fetched from Google when it is empty, so an
|
||||
account that connected during a hiccup corrects itself on the next look
|
||||
rather than showing an empty panel for ever.
|
||||
responses:
|
||||
"200": { $ref: "#/components/responses/Calendar" }
|
||||
delete:
|
||||
tags: [integrations]
|
||||
summary: Disconnect the calendar and forget its tokens.
|
||||
description: >
|
||||
The cached events go too. Unlike the tasks a mailbox made, none of this is
|
||||
the person's own work — it is a copy of somebody else's calendar, and it
|
||||
means nothing without the account.
|
||||
responses:
|
||||
"204": { description: Disconnected. }
|
||||
|
||||
/api/v1/calendar/calendars:
|
||||
put:
|
||||
tags: [integrations]
|
||||
summary: Choose which calendars are drawn.
|
||||
description: >
|
||||
The whole set in one write rather than one flip at a time, for the reason
|
||||
the projects' order is written the same way: the question is "these ones",
|
||||
and a list built out of several requests can land half applied. A calendar
|
||||
turned off has its cached events removed; they come back if it is turned
|
||||
on again.
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
shown:
|
||||
type: array
|
||||
items: { type: string }
|
||||
responses:
|
||||
"204": { description: Saved. }
|
||||
"404": { $ref: "#/components/responses/NotFound" }
|
||||
|
||||
/api/v1/calendar/authorize:
|
||||
post:
|
||||
tags: [integrations]
|
||||
summary: Begin the OAuth flow for a calendar.
|
||||
description: |
|
||||
The same registration Gmail signs in through, asking for
|
||||
`calendar.readonly` and coming back to `/oauth/calendar/callback`. Google
|
||||
issues a refresh token per authorisation, so connecting a calendar does not
|
||||
disturb a working Gmail connection.
|
||||
responses:
|
||||
"200":
|
||||
description: OK
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
url: { type: string }
|
||||
"409":
|
||||
$ref: "#/components/responses/Conflict"
|
||||
|
||||
/api/v1/calendar/sync:
|
||||
post:
|
||||
tags: [integrations]
|
||||
summary: Refresh the chosen calendars now.
|
||||
description: >
|
||||
Refreshes the list of calendars and replaces the cached window of events
|
||||
for the ones that are shown. `partial` means the budget ran out before the
|
||||
last calendar; what was read is kept, and the rest arrive on the next run.
|
||||
responses:
|
||||
"200":
|
||||
description: OK
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
events: { type: integer }
|
||||
partial: { type: boolean }
|
||||
"502": { $ref: "#/components/responses/Upstream" }
|
||||
|
||||
/api/v1/calendar/events:
|
||||
get:
|
||||
tags: [integrations]
|
||||
summary: The events of the chosen calendars, between two days.
|
||||
description: |
|
||||
Both ends inclusive, and an event that merely overlaps the range comes
|
||||
back — a grid asks which cells an event covers, not where it begins.
|
||||
|
||||
`from` and `to` in the answer are the window verdande holds a copy of,
|
||||
which is not the window that was asked for. A grid paged past it gets no
|
||||
events, and an empty answer with no explanation is a calendar quietly
|
||||
claiming a day is clear.
|
||||
parameters:
|
||||
- { name: from, in: query, required: true, schema: { type: string, format: date } }
|
||||
- { name: to, in: query, required: true, schema: { type: string, format: date } }
|
||||
responses:
|
||||
"200":
|
||||
description: OK
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
events:
|
||||
type: array
|
||||
items: { $ref: "#/components/schemas/CalendarEvent" }
|
||||
from: { type: string, format: date }
|
||||
to: { type: string, format: date }
|
||||
"422": { $ref: "#/components/responses/Validation" }
|
||||
|
||||
/api/v1/export/notes.zip:
|
||||
get:
|
||||
tags: [notes]
|
||||
summary: Every note as a Markdown file, in a zip.
|
||||
description: >
|
||||
One file per note, named after its title, containing exactly what is
|
||||
stored. Nothing is converted on the way out — the note on disk is already
|
||||
the file you would export — so it opens in Obsidian, in a text editor, or
|
||||
in anything else that reads Markdown.
|
||||
responses:
|
||||
"200":
|
||||
description: A zip archive.
|
||||
content:
|
||||
application/zip:
|
||||
schema: { type: string, format: binary }
|
||||
|
||||
/api/v1/notes:
|
||||
get:
|
||||
tags: [notes]
|
||||
summary: Notes — listed, narrowed to a project, or searched.
|
||||
description: >
|
||||
With no parameters: the notes filed under no project, which are the
|
||||
person's own. With `project`: that project's, which needs read access to
|
||||
it. With `q`: a full-text search across every note they can see, matching
|
||||
Danish spelling either way round — "grøn" finds "gron" and the reverse.
|
||||
parameters:
|
||||
- name: q
|
||||
in: query
|
||||
schema: { type: string }
|
||||
- name: project
|
||||
in: query
|
||||
schema: { type: string }
|
||||
responses:
|
||||
"200":
|
||||
description: OK
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
notes:
|
||||
type: array
|
||||
items: { $ref: "#/components/schemas/Note" }
|
||||
post:
|
||||
tags: [notes]
|
||||
summary: Write a note.
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema: { $ref: "#/components/schemas/NoteInput" }
|
||||
responses:
|
||||
"201":
|
||||
description: Written
|
||||
content:
|
||||
application/json:
|
||||
schema: { $ref: "#/components/schemas/Note" }
|
||||
"403": { $ref: "#/components/responses/Forbidden" }
|
||||
|
||||
/api/v1/notes/import:
|
||||
post:
|
||||
tags: [notes]
|
||||
summary: A zip of Markdown files becomes notes.
|
||||
description: >
|
||||
The mirror of the export, and the way in from anywhere else: a folder of
|
||||
Markdown is what Obsidian, Bear, iA Writer and a shell script all produce.
|
||||
Filenames are ignored — the title is the first line of the body, here as
|
||||
everywhere, and taking it from the filename would be a second rule for the
|
||||
same thing. Directories, macOS resource forks and anything that is not .md
|
||||
are passed over.
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
multipart/form-data:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
file: { type: string, format: binary }
|
||||
responses:
|
||||
"200":
|
||||
description: Read
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
created: { type: integer }
|
||||
skipped: { type: integer, description: Files that were not usable as a note. }
|
||||
"413": { description: The archive is too large. }
|
||||
|
||||
/api/v1/notes/{noteID}:
|
||||
parameters:
|
||||
- name: noteID
|
||||
in: path
|
||||
required: true
|
||||
schema: { type: string }
|
||||
get:
|
||||
tags: [notes]
|
||||
summary: One note.
|
||||
responses:
|
||||
"200":
|
||||
description: OK
|
||||
content:
|
||||
application/json:
|
||||
schema: { $ref: "#/components/schemas/Note" }
|
||||
"404": { $ref: "#/components/responses/NotFound" }
|
||||
patch:
|
||||
tags: [notes]
|
||||
summary: Change a note. Every field is optional; what is sent is what changes.
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema: { $ref: "#/components/schemas/NoteInput" }
|
||||
responses:
|
||||
"200":
|
||||
description: OK
|
||||
content:
|
||||
application/json:
|
||||
schema: { $ref: "#/components/schemas/Note" }
|
||||
"404": { $ref: "#/components/responses/NotFound" }
|
||||
delete:
|
||||
tags: [notes]
|
||||
summary: Put a note in the trash. It can be brought back.
|
||||
responses:
|
||||
"204": { description: Deleted }
|
||||
"404": { $ref: "#/components/responses/NotFound" }
|
||||
|
||||
/api/v1/notes/linking/{kind}/{targetID}:
|
||||
get:
|
||||
tags: [notes]
|
||||
summary: What has been written about this.
|
||||
description: >
|
||||
The backwards direction. A task's id gives every note that mentions it,
|
||||
a project's name gives every note carrying that #tag.
|
||||
parameters:
|
||||
- name: kind
|
||||
in: path
|
||||
required: true
|
||||
schema: { type: string, enum: [task, note, project] }
|
||||
- name: targetID
|
||||
in: path
|
||||
required: true
|
||||
schema: { type: string }
|
||||
responses:
|
||||
"200":
|
||||
description: OK
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
notes:
|
||||
type: array
|
||||
items: { $ref: "#/components/schemas/Note" }
|
||||
"400": { $ref: "#/components/responses/BadRequest" }
|
||||
|
||||
/api/v1/mailboxes:
|
||||
get:
|
||||
tags: [integrations]
|
||||
@@ -2963,6 +3587,26 @@ paths:
|
||||
responses:
|
||||
"302": { description: "Back into the app, with the outcome as a query parameter." }
|
||||
|
||||
/oauth/calendar/callback:
|
||||
get:
|
||||
tags: [integrations]
|
||||
summary: Where Google sends the browser after consent to a calendar.
|
||||
description: |
|
||||
A URI of its own rather than a reuse of Gmail's. One callback deciding from
|
||||
stored state which of the two flows had come back would put the choice of
|
||||
which token store to write behind a lookup — and when that goes wrong it
|
||||
goes wrong by overwriting a working Gmail connection with calendar tokens.
|
||||
|
||||
Answers with a redirect to `/indstillinger/integrationer`, not with JSON.
|
||||
An `org_internal` error here means the OAuth app is registered as Internal
|
||||
and the account is outside the organisation.
|
||||
parameters:
|
||||
- { name: code, in: query, schema: { type: string } }
|
||||
- { name: state, in: query, schema: { type: string } }
|
||||
- { name: error, in: query, schema: { type: string } }
|
||||
responses:
|
||||
"302": { description: "Back into the app, with the outcome as a query parameter." }
|
||||
|
||||
/mcp:
|
||||
post:
|
||||
tags: [integrations]
|
||||
@@ -3145,6 +3789,40 @@ components:
|
||||
email: { type: string }
|
||||
trigger: { type: string, enum: ["", starred, label, both] }
|
||||
label: { type: string }
|
||||
Calendar:
|
||||
description: OK
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
type: object
|
||||
properties:
|
||||
connected: { type: boolean }
|
||||
account:
|
||||
type: string
|
||||
description: >
|
||||
The address of the connected account, taken from the id of its
|
||||
primary calendar — which is where Google puts it, so no profile
|
||||
scope has to be asked for to learn it.
|
||||
read_only:
|
||||
type: boolean
|
||||
description: >
|
||||
Always true today. Stated rather than assumed: an interface that
|
||||
quietly looks editable is one somebody will try to edit.
|
||||
has_client:
|
||||
type: boolean
|
||||
description: >
|
||||
Whether an OAuth client is registered on this instance at all, so
|
||||
the page can explain rather than offer a button that answers 409.
|
||||
redirect_uri:
|
||||
type: string
|
||||
description: >
|
||||
What has to be registered with Google, spelled out. Derived from
|
||||
VERDANDE_BASE_URL and matched exactly — the single most likely
|
||||
thing to be wrong, and the error Google gives names neither value.
|
||||
last_sync_at: { type: string, format: date-time }
|
||||
calendars:
|
||||
type: array
|
||||
items: { $ref: "#/components/schemas/Calendar" }
|
||||
ImportResult:
|
||||
description: Created
|
||||
content:
|
||||
@@ -3162,6 +3840,89 @@ components:
|
||||
items: { type: string }
|
||||
|
||||
schemas:
|
||||
Calendar:
|
||||
type: object
|
||||
description: >
|
||||
One calendar inside a connected account. Kept whether or not it is shown,
|
||||
so turning one off and on again gives back the same row.
|
||||
properties:
|
||||
id: { type: string }
|
||||
remote_id: { type: string, description: "The id Google knows it by; an address for the primary calendar." }
|
||||
name: { type: string }
|
||||
colour:
|
||||
type: string
|
||||
description: >
|
||||
Google's own hex value, not one of verdande's ten named colours. Those
|
||||
are names so the interface can restyle them per theme and they are
|
||||
verdande's palette; a Google calendar is told apart by the colour the
|
||||
person already knows it by.
|
||||
time_zone: { type: string }
|
||||
primary: { type: boolean }
|
||||
writable:
|
||||
type: boolean
|
||||
description: >
|
||||
Whether the connected account could write to it. Nothing does — the
|
||||
scope is read-only — and it is stored so the day something can, the
|
||||
interface knows which calendars could even be offered.
|
||||
shown: { type: boolean }
|
||||
|
||||
CalendarEvent:
|
||||
type: object
|
||||
description: >
|
||||
One occurrence, in the shape a grid asks about. A recurrence is expanded by
|
||||
Google into its occurrences, so what arrives here is a day, not a rule.
|
||||
properties:
|
||||
id: { type: string }
|
||||
calendar_id: { type: string }
|
||||
summary: { type: string }
|
||||
starts_at:
|
||||
type: string
|
||||
description: >
|
||||
RFC 3339 as Google wrote it, offset and all. Empty for an all-day
|
||||
event, which has a day rather than a moment — inventing midnight for it
|
||||
is how an event moves a day when a container moves time zone.
|
||||
ends_at: { type: string }
|
||||
start_day: { type: string, format: date }
|
||||
end_day:
|
||||
type: string
|
||||
format: date
|
||||
description: >
|
||||
The last day the event covers, inclusive. Google's all-day end date is
|
||||
exclusive and is corrected on the way in, once, rather than by every
|
||||
reader.
|
||||
all_day: { type: boolean }
|
||||
location: { type: string }
|
||||
url: { type: string, description: "Opens the event in Google's own interface." }
|
||||
calendar_name: { type: string }
|
||||
colour: { type: string }
|
||||
|
||||
Note:
|
||||
type: object
|
||||
description: A page of Markdown. The body is the stored form and the exported form.
|
||||
properties:
|
||||
id: { type: string }
|
||||
project_id: { type: string }
|
||||
title: { type: string, readOnly: true, description: "The note's first line. Derived on every save; it cannot be set." }
|
||||
body: { type: string, description: Markdown }
|
||||
pinned: { type: boolean }
|
||||
created_at: { type: string, format: date-time }
|
||||
updated_at: { type: string, format: date-time }
|
||||
links:
|
||||
type: array
|
||||
description: What the body points at, read out of the text on the way in.
|
||||
items:
|
||||
type: object
|
||||
properties:
|
||||
kind: { type: string, enum: [task, note, project] }
|
||||
target_id: { type: string }
|
||||
|
||||
NoteInput:
|
||||
type: object
|
||||
properties:
|
||||
project_id: { type: string, description: "Empty files it under no project." }
|
||||
body: { type: string, description: "Markdown. Its first line becomes the title." }
|
||||
pinned: { type: boolean }
|
||||
|
||||
Mailbox:
|
||||
type: object
|
||||
description: One connected mailbox. The password and any token are never returned.
|
||||
|
||||
|
Before Width: | Height: | Size: 166 KiB After Width: | Height: | Size: 239 KiB |
|
Before Width: | Height: | Size: 181 KiB After Width: | Height: | Size: 276 KiB |
|
Before Width: | Height: | Size: 91 KiB After Width: | Height: | Size: 136 KiB |
|
Before Width: | Height: | Size: 228 KiB After Width: | Height: | Size: 306 KiB |
|
Before Width: | Height: | Size: 243 KiB After Width: | Height: | Size: 241 KiB |
|
After Width: | Height: | Size: 328 KiB |
|
After Width: | Height: | Size: 308 KiB |
|
Before Width: | Height: | Size: 176 KiB After Width: | Height: | Size: 215 KiB |
|
Before Width: | Height: | Size: 230 KiB After Width: | Height: | Size: 240 KiB |
|
Before Width: | Height: | Size: 199 KiB After Width: | Height: | Size: 232 KiB |
|
Before Width: | Height: | Size: 154 KiB After Width: | Height: | Size: 222 KiB |
|
Before Width: | Height: | Size: 153 KiB After Width: | Height: | Size: 221 KiB |
|
Before Width: | Height: | Size: 188 KiB After Width: | Height: | Size: 209 KiB |
|
Before Width: | Height: | Size: 167 KiB After Width: | Height: | Size: 180 KiB |
@@ -19,6 +19,8 @@ import (
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
|
||||
"github.com/kristianwind/verdande/internal/safedial"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
@@ -76,7 +78,16 @@ func New(cfg Config) *Client {
|
||||
// Generous: a local model on modest hardware can take a while to produce
|
||||
// its first token, and the alternative is a timeout that only ever fires
|
||||
// for the people running their own.
|
||||
http: &http.Client{Timeout: 120 * time.Second},
|
||||
// safedial, not a plain client. `base_url` is typed into a field in the
|
||||
// interface, so it is a user-supplied address — and a server that fetches a
|
||||
// user-supplied address is a way to ask it to reach things the caller
|
||||
// cannot: the panel next door, a database on the same bridge, the cloud
|
||||
// metadata endpoint. On a homelab that is most of what is worth reaching.
|
||||
//
|
||||
// The check is on the resolved address rather than on the URL, because a
|
||||
// name answers whatever its owner says — and can answer differently the
|
||||
// second time, after a parse-time check has passed.
|
||||
http: safedial.Client(120 * time.Second),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -268,11 +279,35 @@ func (c *Client) do(req *http.Request, out any) error {
|
||||
// The provider's own message is included: for a local model it is usually
|
||||
// the actual problem ("model not found"), and hiding it would leave the
|
||||
// operator with nothing to go on.
|
||||
return fmt.Errorf("ai: %s said %s: %s", req.URL.Host, resp.Status, truncate(string(raw), 300))
|
||||
// The provider's body is no longer passed back to the caller.
|
||||
//
|
||||
// It was, and combined with a free-text base URL that made a working
|
||||
// internal scanner: point it at a host, read the first three hundred bytes
|
||||
// of whatever answered. The operator still needs the message — for a local
|
||||
// model it is usually the actual problem — so it goes to the log, where it
|
||||
// is available to the person running the instance and to nobody else.
|
||||
return &upstreamError{Host: req.URL.Host, Status: resp.Status, Body: truncate(string(raw), 300)}
|
||||
}
|
||||
return json.Unmarshal(raw, out)
|
||||
}
|
||||
|
||||
// upstreamError keeps the provider's own words for the log and away from the
|
||||
// response. Error() is what reaches a caller; Detail() is what is written down.
|
||||
type upstreamError struct {
|
||||
Host string
|
||||
Status string
|
||||
Body string
|
||||
}
|
||||
|
||||
func (e *upstreamError) Error() string {
|
||||
return fmt.Sprintf("ai: %s said %s", e.Host, e.Status)
|
||||
}
|
||||
|
||||
// Detail is the whole of it, for the instance's own error log.
|
||||
func (e *upstreamError) Detail() string {
|
||||
return fmt.Sprintf("ai: %s said %s: %s", e.Host, e.Status, e.Body)
|
||||
}
|
||||
|
||||
func truncate(s string, max int) string {
|
||||
if len(s) <= max {
|
||||
return s
|
||||
|
||||
@@ -0,0 +1,139 @@
|
||||
// Package beacon answers one question for the project, and refuses to answer any
|
||||
// other: how many installations are out there.
|
||||
//
|
||||
// It sends two values once a day — an anonymous id the installation made up for
|
||||
// itself, and the version it is running — and nothing else. No IP is stored by
|
||||
// the collector, no hostname, no domain, no account, no usage. The payload is
|
||||
// small enough to be printed in full on the settings page, and it is, because a
|
||||
// promise about telemetry is worth exactly as much as the reader's ability to
|
||||
// check it.
|
||||
//
|
||||
// # On by default, and off in one click
|
||||
//
|
||||
// The alternative that was asked for first was mandatory, and the argument
|
||||
// against it is practical rather than principled: it cannot be enforced. The
|
||||
// domain can be blocked, the binary can be patched, and a fork removes it in its
|
||||
// first commit — so mandatory buys nothing over on-by-default, while costing the
|
||||
// trust the rest of this program works for. Secrets are sealed here, the key is
|
||||
// kept out of the database, and the backup page warns you what a copy contains.
|
||||
// Telemetry you are not allowed to refuse is the one sentence in that story that
|
||||
// does not fit.
|
||||
//
|
||||
// So: on by default, said plainly, one switch to turn it off. That collects
|
||||
// essentially the same number and leaves nobody feeling tricked.
|
||||
//
|
||||
// # Where this disagrees with the rest of the program
|
||||
//
|
||||
// `config.UpdateCheck` is off unless asked for, and its comment gives the reason:
|
||||
// "a self-hosted app that reaches out without being told to has broken the deal
|
||||
// its operator made by self-hosting." That is the same situation as this one, and
|
||||
// the two defaults do not agree.
|
||||
//
|
||||
// The disagreement is written down here rather than smoothed over, because it is
|
||||
// a real one and the reader deserves to see both halves. The case for treating
|
||||
// them differently is that the update check reveals an instance to a third party
|
||||
// (GitHub) and returns something the operator can act on, while this returns
|
||||
// nothing and goes to the project itself — and that a count nobody sends is a
|
||||
// count that does not exist, whereas an update nobody checks for is one you can
|
||||
// still go and look up. Whether that is a distinction or an excuse is a judgement
|
||||
// call, and it was made deliberately by the person whose instance this is.
|
||||
//
|
||||
// If it is ever decided the other way, the change is one word: the default in
|
||||
// `beaconSettings`. Nothing else in this package assumes which way it points.
|
||||
//
|
||||
// # What the collector can still see
|
||||
//
|
||||
// A request has a source address whether or not anybody writes it down. The
|
||||
// collector does not store or log it, and the code is arranged so that it never
|
||||
// reaches a function that could — but it passes through a reverse proxy and a
|
||||
// tunnel on the way, and those keep their own logs. Saying "we send no IP" would
|
||||
// be a half-truth; what is true is that nothing here records one.
|
||||
package beacon
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"regexp"
|
||||
"time"
|
||||
)
|
||||
|
||||
// DefaultCollector is where a fresh installation reports, unless it is told
|
||||
// otherwise. Somebody running their own fleet points it at their own instance;
|
||||
// the field is on the settings page for exactly that.
|
||||
const DefaultCollector = "https://verdande.yggdrasilpanel.com/api/v1/beacon"
|
||||
|
||||
// Payload is the entire message. Two fields, and adding a third is a change to
|
||||
// what this program promises, not a change to a struct.
|
||||
type Payload struct {
|
||||
InstanceID string `json:"instance_id"`
|
||||
Version string `json:"version"`
|
||||
}
|
||||
|
||||
// InstanceID and Version are checked on the way in as well as written on the way
|
||||
// out, because the collector's endpoint is unauthenticated: anything that reaches
|
||||
// it is a stranger's bytes, and it should be able to store only the two shapes it
|
||||
// expects.
|
||||
var (
|
||||
idShape = regexp.MustCompile(`^[0-9a-fA-F-]{16,64}$`)
|
||||
versionShape = regexp.MustCompile(`^v?[0-9]{1,4}(\.[0-9]{1,5}){0,3}(-[0-9A-Za-z.]{1,20})?$`)
|
||||
)
|
||||
|
||||
// ValidID reports whether an id is the shape this program writes.
|
||||
func ValidID(s string) bool { return idShape.MatchString(s) }
|
||||
|
||||
// ValidVersion allows a version number and the word this binary uses when it was
|
||||
// built outside a release.
|
||||
func ValidVersion(s string) bool { return s == "dev" || versionShape.MatchString(s) }
|
||||
|
||||
// Send delivers one ping. It is deliberately dull: a short timeout, no retry, no
|
||||
// queue, and a failure that is returned rather than made anybody's problem.
|
||||
//
|
||||
// No retry, because the next pass is tomorrow and a missed day changes a count of
|
||||
// installations by nothing. A telemetry ping that retries is a telemetry ping
|
||||
// that can hammer somebody's server on the day it is down.
|
||||
func Send(ctx context.Context, client *http.Client, collector string, p Payload) error {
|
||||
if !ValidID(p.InstanceID) {
|
||||
return fmt.Errorf("beacon: refusing to send a malformed id")
|
||||
}
|
||||
// Versionen kontrolleres også her, ikke kun hos modtageren. Kommentaren ved
|
||||
// mønstrene lovede begge veje, og gjorde det ikke — og en installation, der
|
||||
// sender noget mærkeligt, skal opdage det hos sig selv frem for at være noget,
|
||||
// en fremmed collector skal filtrere fra.
|
||||
if !ValidVersion(p.Version) {
|
||||
return fmt.Errorf("beacon: refusing to send a malformed version")
|
||||
}
|
||||
body, err := json.Marshal(p)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(ctx, 15*time.Second)
|
||||
defer cancel()
|
||||
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, collector, bytes.NewReader(body))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
// A plain user agent with the version already in the payload. Nothing is
|
||||
// learned from it that the body does not already say, and it makes the request
|
||||
// legible in a log rather than anonymous-looking.
|
||||
req.Header.Set("User-Agent", "verdande-beacon/1")
|
||||
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
// Read and discard, so the connection can be reused rather than dropped.
|
||||
_, _ = io.Copy(io.Discard, io.LimitReader(resp.Body, 4<<10))
|
||||
|
||||
if resp.StatusCode >= 400 {
|
||||
return fmt.Errorf("beacon: collector answered %d", resp.StatusCode)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,114 @@
|
||||
package beacon
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// Hvad der bliver sendt, tegn for tegn.
|
||||
//
|
||||
// Prøven er skrevet imod *hele* kroppen og ikke imod enkelte felter, fordi det er
|
||||
// den, der er løftet: siden siger "præcis det her og ikke mere", og et felt lagt
|
||||
// til senere skal vælte noget. En prøve, der kun så efter, at instance_id var med,
|
||||
// ville lade en ny nøgle glide igennem uden at nogen opdagede det.
|
||||
func TestSendsExactlyTwoFields(t *testing.T) {
|
||||
var got map[string]any
|
||||
var contentType string
|
||||
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
contentType = r.Header.Get("Content-Type")
|
||||
body, _ := io.ReadAll(r.Body)
|
||||
if err := json.Unmarshal(body, &got); err != nil {
|
||||
t.Errorf("kroppen er ikke JSON: %v", err)
|
||||
}
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
err := Send(context.Background(), srv.Client(), srv.URL, Payload{
|
||||
InstanceID: "01a01b16-4e54-7000-9eaa-2f8eb0629c3a",
|
||||
Version: "v0.26.3",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("Send: %v", err)
|
||||
}
|
||||
|
||||
if contentType != "application/json" {
|
||||
t.Errorf("Content-Type = %q", contentType)
|
||||
}
|
||||
if len(got) != 2 {
|
||||
t.Fatalf("der blev sendt %d felter, ikke to: %v", len(got), got)
|
||||
}
|
||||
if got["instance_id"] != "01a01b16-4e54-7000-9eaa-2f8eb0629c3a" {
|
||||
t.Errorf("instance_id = %v", got["instance_id"])
|
||||
}
|
||||
if got["version"] != "v0.26.3" {
|
||||
t.Errorf("version = %v", got["version"])
|
||||
}
|
||||
}
|
||||
|
||||
// En collector, der er nede, er ikke denne installations problem.
|
||||
func TestSendReportsAFailureRatherThanRetrying(t *testing.T) {
|
||||
calls := 0
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
calls++
|
||||
w.WriteHeader(http.StatusInternalServerError)
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
err := Send(context.Background(), srv.Client(), srv.URL, Payload{
|
||||
InstanceID: "01a01b16-4e54-7000-9eaa-2f8eb0629c3a", Version: "v1.0.0",
|
||||
})
|
||||
if err == nil {
|
||||
t.Fatal("en 500 skal give en fejl")
|
||||
}
|
||||
if calls != 1 {
|
||||
t.Errorf("der blev prøvet %d gange; en telemetri-ping må ikke hamre på en server, der er nede", calls)
|
||||
}
|
||||
}
|
||||
|
||||
// Et id, der ikke er id-formet, sendes slet ikke — så en fejl her ikke bliver til
|
||||
// noget, en fremmed collector skal filtrere fra.
|
||||
func TestSendRefusesAMalformedID(t *testing.T) {
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
t.Error("der blev sendt noget, som ikke skulle sendes")
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
if err := Send(context.Background(), srv.Client(), srv.URL, Payload{
|
||||
InstanceID: "; DROP TABLE beacon_installs", Version: "v1.0.0",
|
||||
}); err == nil {
|
||||
t.Fatal("et misformet id skal afvises før det forlader maskinen")
|
||||
}
|
||||
}
|
||||
|
||||
func TestShapes(t *testing.T) {
|
||||
ok := []string{"01a01b16-4e54-7000-9eaa-2f8eb0629c3a", "89df44d7-3742-4611-9a43-db0b07733c6d"}
|
||||
for _, s := range ok {
|
||||
if !ValidID(s) {
|
||||
t.Errorf("ValidID(%q) = false", s)
|
||||
}
|
||||
}
|
||||
bad := []string{"", "kort", "../../etc/passwd", "01a01b16-4e54-7000-9eaa-2f8eb0629c3a'; --",
|
||||
"<script>alert(1)</script>"}
|
||||
for _, s := range bad {
|
||||
if ValidID(s) {
|
||||
t.Errorf("ValidID(%q) = true", s)
|
||||
}
|
||||
}
|
||||
|
||||
for _, s := range []string{"v0.26.3", "0.26.3", "v1.2.3-rc.1", "dev"} {
|
||||
if !ValidVersion(s) {
|
||||
t.Errorf("ValidVersion(%q) = false", s)
|
||||
}
|
||||
}
|
||||
for _, s := range []string{"", "v<script>", "v" + string(make([]byte, 200))} {
|
||||
if ValidVersion(s) {
|
||||
t.Errorf("ValidVersion(%q) = true", s)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -59,10 +59,19 @@ type Config struct {
|
||||
// without spending the real one on every run.
|
||||
GmailSyncBudget time.Duration
|
||||
|
||||
// GmailEndpoints redirects the Gmail calls at a server the tests control. Empty
|
||||
// in production, where it means Google.
|
||||
GmailTokenURL string
|
||||
GmailAPIURL string
|
||||
// CalendarSyncBudget is the same bound for one calendar refresh, and exists for
|
||||
// the same reason: a slow Google held the request open past what Cloudflare is
|
||||
// willing to wait, and the browser got Cloudflare's HTML instead of an error
|
||||
// this server could explain.
|
||||
CalendarSyncBudget time.Duration
|
||||
|
||||
// Where Google is. Empty in production; a test points them at a server it
|
||||
// controls, which is the only way to exercise a Google that is slow or refuses.
|
||||
// The token endpoint is shared — Gmail and Calendar sign in through the same
|
||||
// registration — and the two APIs are not.
|
||||
GoogleTokenURL string
|
||||
GmailAPIURL string
|
||||
CalendarAPIURL string
|
||||
|
||||
PanelURL string
|
||||
PanelToken string
|
||||
@@ -98,14 +107,15 @@ func Load() (*Config, error) {
|
||||
TrashRetention: 30 * 24 * time.Hour,
|
||||
Dev: envBool("VERDANDE_DEV", false),
|
||||
|
||||
GmailClientID: env("VERDANDE_GMAIL_CLIENT_ID", ""),
|
||||
GmailSyncBudget: 25 * time.Second,
|
||||
PanelURL: strings.TrimSuffix(env("VERDANDE_PANEL_URL", ""), "/"),
|
||||
PanelToken: env("VERDANDE_PANEL_TOKEN", ""),
|
||||
PanelServerID: env("VERDANDE_PANEL_SERVER_ID", ""),
|
||||
GmailClientSecret: env("VERDANDE_GMAIL_CLIENT_SECRET", ""),
|
||||
SecretKey: env("VERDANDE_SECRET_KEY", ""),
|
||||
UpdateCheck: envBool("VERDANDE_UPDATE_CHECK", false),
|
||||
GmailClientID: env("VERDANDE_GMAIL_CLIENT_ID", ""),
|
||||
GmailSyncBudget: 25 * time.Second,
|
||||
CalendarSyncBudget: 25 * time.Second,
|
||||
PanelURL: strings.TrimSuffix(env("VERDANDE_PANEL_URL", ""), "/"),
|
||||
PanelToken: env("VERDANDE_PANEL_TOKEN", ""),
|
||||
PanelServerID: env("VERDANDE_PANEL_SERVER_ID", ""),
|
||||
GmailClientSecret: env("VERDANDE_GMAIL_CLIENT_SECRET", ""),
|
||||
SecretKey: env("VERDANDE_SECRET_KEY", ""),
|
||||
UpdateCheck: envBool("VERDANDE_UPDATE_CHECK", false),
|
||||
}
|
||||
|
||||
var err error
|
||||
@@ -152,6 +162,15 @@ func (c *Config) BackupsDir() string { return filepath.Join(c.DataDir, "backups"
|
||||
// configured separately and left to drift out of step with it.
|
||||
func (c *Config) GmailRedirectURL() string { return c.BaseURL + "/oauth/gmail/callback" }
|
||||
|
||||
// CalendarRedirectURL is a second registered URI rather than a reuse of Gmail's.
|
||||
//
|
||||
// One callback that decided from server-side state which of the two flows had come
|
||||
// back would save the operator one line in Google Cloud, and would put the choice
|
||||
// of *which token store to write* behind a lookup. When that goes wrong it goes
|
||||
// wrong by overwriting a working Gmail connection with calendar tokens — a failure
|
||||
// nobody would think to look for. Two paths cannot do that.
|
||||
func (c *Config) CalendarRedirectURL() string { return c.BaseURL + "/oauth/calendar/callback" }
|
||||
|
||||
func env(key, def string) string {
|
||||
if v, ok := os.LookupEnv(key); ok && v != "" {
|
||||
return v
|
||||
|
||||
@@ -60,6 +60,20 @@ func Parse(query string, ctx Context) (Compiled, error) {
|
||||
if len(p.tokens) == 0 {
|
||||
return Compiled{}, fmt.Errorf("filteret er tomt")
|
||||
}
|
||||
// Refused before it is parsed, not while.
|
||||
//
|
||||
// `combine` rebuilds the whole accumulated SQL string and re-copies the
|
||||
// argument slice per operator, so the cost is quadratic in the number of
|
||||
// terms: fifty thousand of them — three hundred kilobytes, which fits in a
|
||||
// query string — took seven seconds of CPU and forty-six gigabytes of
|
||||
// allocation in one request. The preview endpoint runs this on whatever is
|
||||
// typed, and there is no write timeout on the server to cut it off.
|
||||
//
|
||||
// Two hundred terms is a filter nobody has ever written and far below where it
|
||||
// costs anything.
|
||||
if len(p.tokens) > maxTokens {
|
||||
return Compiled{}, fmt.Errorf("filteret er for langt")
|
||||
}
|
||||
|
||||
node, err := p.parseExpression()
|
||||
if err != nil {
|
||||
@@ -140,10 +154,19 @@ func isOperator(r rune) bool {
|
||||
|
||||
// --- parser -------------------------------------------------------------------
|
||||
|
||||
// The ceilings on one filter. See Parse for why.
|
||||
const (
|
||||
maxTokens = 200
|
||||
maxDepth = 32
|
||||
)
|
||||
|
||||
type parser struct {
|
||||
tokens []token
|
||||
pos int
|
||||
ctx Context
|
||||
// depth guards against a filter that is only parentheses. Each level is a
|
||||
// stack frame, and nothing else stops somebody sending a megabyte of them.
|
||||
depth int
|
||||
}
|
||||
|
||||
func (p *parser) done() bool { return p.pos >= len(p.tokens) }
|
||||
@@ -187,6 +210,13 @@ func (p *parser) parseTerm() (Compiled, error) {
|
||||
if p.done() {
|
||||
return Compiled{}, fmt.Errorf("filteret slutter midt i et udtryk")
|
||||
}
|
||||
// Nesting is stack depth. A filter made of half a million opening parentheses
|
||||
// is a megabyte of text and a stack frame for each.
|
||||
p.depth++
|
||||
defer func() { p.depth-- }()
|
||||
if p.depth > maxDepth {
|
||||
return Compiled{}, fmt.Errorf("filteret har for mange niveauer")
|
||||
}
|
||||
|
||||
switch t := p.peek(); t.kind {
|
||||
case tokNot:
|
||||
|
||||
@@ -0,0 +1,339 @@
|
||||
// Package gcal reads a Google Calendar. It never writes one.
|
||||
//
|
||||
// The scope is calendar.readonly, and that is the whole of the decision: verdande
|
||||
// already is a CalDAV server, so a Google event shown here is a second calendar
|
||||
// laid over the first, not a copy Google and verdande now both have to agree
|
||||
// about. Writing needs `calendar.events`, and it needs an answer to what happens
|
||||
// when both sides moved the same meeting — which is a synchronisation model, not a
|
||||
// scope. Read first; write later and to one calendar at a time.
|
||||
//
|
||||
// What the API gives back and what this returns are deliberately different shapes.
|
||||
// Google answers with RFC 3339 instants for a timed event and bare dates for an
|
||||
// all-day one, and with an *exclusive* end date for the second. A calendar grid
|
||||
// asks one question — "which cells does this event cover" — so that is what an
|
||||
// Event answers, in days, worked out once here rather than in every caller.
|
||||
package gcal
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/kristianwind/verdande/internal/google"
|
||||
)
|
||||
|
||||
// Scope is read-only, and only the calendar.
|
||||
//
|
||||
// Google also offers calendar.calendarlist.readonly and calendar.events.readonly,
|
||||
// which together are strictly less than this — they leave out access rules and
|
||||
// account settings, which verdande has no use for. They are the tighter pair and
|
||||
// worth moving to; they are also newer, and this one has been documented and
|
||||
// accepted unchanged for a decade. A consent screen that fails with `invalid_scope`
|
||||
// is not a place to be adventurous on somebody else's Google account.
|
||||
const Scope = "https://www.googleapis.com/auth/calendar.readonly"
|
||||
|
||||
const defaultAPIBase = "https://www.googleapis.com/calendar/v3"
|
||||
|
||||
// The OAuth flow, which is Google's rather than the calendar's — the same one
|
||||
// Gmail signs in through, asking for the scope above. Aliased so a caller reads
|
||||
// one package for one feature.
|
||||
type (
|
||||
Config = google.Config
|
||||
PKCE = google.PKCE
|
||||
Token = google.Token
|
||||
)
|
||||
|
||||
var NewPKCE = google.NewPKCE
|
||||
|
||||
// ErrUnauthorized means the tokens are no longer good — access was revoked, or the
|
||||
// password changed. Re-exported so a caller need not know the flow lives elsewhere.
|
||||
var ErrUnauthorized = google.ErrUnauthorized
|
||||
|
||||
type Client struct {
|
||||
accessToken string
|
||||
// base is where the Calendar API is. A field rather than a constant so a test
|
||||
// can point it at a server it controls; empty is Google. There is no other way
|
||||
// to exercise a calendar that is slow, empty, or refuses.
|
||||
base string
|
||||
http *http.Client
|
||||
}
|
||||
|
||||
func NewClient(accessToken string) *Client {
|
||||
return &Client{accessToken: accessToken, http: &http.Client{Timeout: 30 * time.Second}}
|
||||
}
|
||||
|
||||
// At points the client at another server. Tests only; the zero value is Google.
|
||||
func (c *Client) At(base string) *Client {
|
||||
c.base = base
|
||||
return c
|
||||
}
|
||||
|
||||
func (c *Client) api() string {
|
||||
if c.base != "" {
|
||||
return c.base
|
||||
}
|
||||
return defaultAPIBase
|
||||
}
|
||||
|
||||
// Calendar is one entry in the person's calendar list.
|
||||
type Calendar struct {
|
||||
ID string
|
||||
// Name is the summary Google holds. A calendar somebody renamed for themselves
|
||||
// carries summaryOverride instead, and that is the name they will look for.
|
||||
Name string
|
||||
// Colour is Google's own hex value for the calendar, which is what makes two
|
||||
// calendars in one grid tellable apart. Their own colour rather than one picked
|
||||
// here: it is the colour they already know the calendar by.
|
||||
Colour string
|
||||
TimeZone string
|
||||
Primary bool
|
||||
// Owner means the connected account can write to it. Nothing writes today —
|
||||
// it is stored so the day something does, the interface knows which calendars
|
||||
// could even be offered.
|
||||
Owner bool
|
||||
}
|
||||
|
||||
// Calendars lists what the connected account can see.
|
||||
func (c *Client) Calendars(ctx context.Context) ([]Calendar, error) {
|
||||
var out []Calendar
|
||||
page := ""
|
||||
// Bounded rather than "until Google stops": a page token that never empties is
|
||||
// a loop that never ends, and the difference between a bug at Google and a
|
||||
// hung sync should not be this server hanging.
|
||||
for range 10 {
|
||||
v := url.Values{}
|
||||
v.Set("minAccessRole", "reader")
|
||||
v.Set("maxResults", "250")
|
||||
if page != "" {
|
||||
v.Set("pageToken", page)
|
||||
}
|
||||
|
||||
var parsed struct {
|
||||
Items []struct {
|
||||
ID string `json:"id"`
|
||||
Summary string `json:"summary"`
|
||||
SummaryOverride string `json:"summaryOverride"`
|
||||
BackgroundColor string `json:"backgroundColor"`
|
||||
TimeZone string `json:"timeZone"`
|
||||
Primary bool `json:"primary"`
|
||||
AccessRole string `json:"accessRole"`
|
||||
Deleted bool `json:"deleted"`
|
||||
} `json:"items"`
|
||||
NextPageToken string `json:"nextPageToken"`
|
||||
}
|
||||
if err := c.get(ctx, c.api()+"/users/me/calendarList?"+v.Encode(), &parsed); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
for _, item := range parsed.Items {
|
||||
if item.Deleted {
|
||||
continue
|
||||
}
|
||||
name := item.SummaryOverride
|
||||
if name == "" {
|
||||
name = item.Summary
|
||||
}
|
||||
out = append(out, Calendar{
|
||||
ID: item.ID, Name: name, Colour: item.BackgroundColor,
|
||||
TimeZone: item.TimeZone, Primary: item.Primary,
|
||||
Owner: item.AccessRole == "owner" || item.AccessRole == "writer",
|
||||
})
|
||||
}
|
||||
if parsed.NextPageToken == "" {
|
||||
break
|
||||
}
|
||||
page = parsed.NextPageToken
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// Event is one occurrence, in the shape a calendar grid asks about.
|
||||
type Event struct {
|
||||
ID string
|
||||
Summary string
|
||||
// StartsAt and EndsAt are RFC 3339 for a timed event and empty for an all-day
|
||||
// one — an all-day event has no instant, only a day, and inventing midnight for
|
||||
// it is how an event moves a day every time a server changes time zone.
|
||||
StartsAt string
|
||||
EndsAt string
|
||||
// StartDay and EndDay are the first and last day the event covers, inclusive at
|
||||
// both ends. Google's all-day end date is exclusive; that is corrected here,
|
||||
// once, rather than in every reader.
|
||||
StartDay string
|
||||
EndDay string
|
||||
AllDay bool
|
||||
Location string
|
||||
// URL opens the event in Google's own interface. Same reasoning as the Gmail
|
||||
// link on a task made from a mail: what verdande shows is a pointer to the
|
||||
// event, not a second copy of it that somebody could try to edit.
|
||||
URL string
|
||||
}
|
||||
|
||||
// Events lists what falls inside a window, one calendar at a time.
|
||||
//
|
||||
// `singleEvents` expands a recurrence into its occurrences, which is the only form
|
||||
// a grid can draw: an RRULE is an instruction, and a Tuesday cell needs the Tuesday
|
||||
// it produced. It also means verdande does not have to grow a second recurrence
|
||||
// engine that agrees with Google's about Danish summer time.
|
||||
//
|
||||
// from and to are dates, inclusive. The window is the caller's business; this
|
||||
// turns it into the half-open interval Google wants.
|
||||
func (c *Client) Events(ctx context.Context, calendarID, from, to string) ([]Event, error) {
|
||||
start, err := time.Parse(dateLayout, from)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("gcal: from: %w", err)
|
||||
}
|
||||
end, err := time.Parse(dateLayout, to)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("gcal: to: %w", err)
|
||||
}
|
||||
|
||||
var out []Event
|
||||
page := ""
|
||||
for range 20 {
|
||||
v := url.Values{}
|
||||
v.Set("singleEvents", "true")
|
||||
v.Set("orderBy", "startTime")
|
||||
v.Set("maxResults", "2500")
|
||||
// UTC, and a day of slack at each end. An event at 23:30 on the last day in
|
||||
// Copenhagen is 21:30 UTC that day, but one at 00:30 on the first day is
|
||||
// 22:30 UTC the day *before* — so a window cut exactly at the requested
|
||||
// dates loses the events at both edges. The extra day is thrown away again
|
||||
// by the day fields below, which are computed from the event's own offset.
|
||||
v.Set("timeMin", start.AddDate(0, 0, -1).Format(time.RFC3339))
|
||||
v.Set("timeMax", end.AddDate(0, 0, 2).Format(time.RFC3339))
|
||||
if page != "" {
|
||||
v.Set("pageToken", page)
|
||||
}
|
||||
|
||||
var parsed struct {
|
||||
Items []rawEvent `json:"items"`
|
||||
NextPageToken string `json:"nextPageToken"`
|
||||
}
|
||||
url := c.api() + "/calendars/" + url.PathEscape(calendarID) + "/events?" + v.Encode()
|
||||
if err := c.get(ctx, url, &parsed); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
for _, item := range parsed.Items {
|
||||
// A cancelled occurrence of a recurring event still comes back, so that
|
||||
// a client keeping its own copy knows to drop it. verdande replaces the
|
||||
// window wholesale, so it only has to not add it.
|
||||
if item.Status == "cancelled" {
|
||||
continue
|
||||
}
|
||||
event, ok := item.event()
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
if event.EndDay < from || event.StartDay > to {
|
||||
continue // the slack above, thrown away
|
||||
}
|
||||
out = append(out, event)
|
||||
}
|
||||
if parsed.NextPageToken == "" {
|
||||
break
|
||||
}
|
||||
page = parsed.NextPageToken
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
const dateLayout = "2006-01-02"
|
||||
|
||||
type rawEvent struct {
|
||||
ID string `json:"id"`
|
||||
Status string `json:"status"`
|
||||
Summary string `json:"summary"`
|
||||
Location string `json:"location"`
|
||||
HTMLLink string `json:"htmlLink"`
|
||||
Start when `json:"start"`
|
||||
End when `json:"end"`
|
||||
}
|
||||
|
||||
type when struct {
|
||||
Date string `json:"date"`
|
||||
DateTime string `json:"dateTime"`
|
||||
}
|
||||
|
||||
// event turns Google's two shapes into the one a grid can read.
|
||||
//
|
||||
// The second return is false for an event with neither a date nor a dateTime —
|
||||
// which Google does return, for an event somebody is still being asked about. It
|
||||
// belongs on no day, so there is no cell to draw it in.
|
||||
func (r rawEvent) event() (Event, bool) {
|
||||
e := Event{
|
||||
ID: r.ID, Summary: strings.TrimSpace(r.Summary),
|
||||
Location: r.Location, URL: r.HTMLLink,
|
||||
}
|
||||
if e.Summary == "" {
|
||||
// Google's own interface says this for an event with no title, and a blank
|
||||
// chip in a grid reads as a rendering fault rather than as an empty title.
|
||||
e.Summary = "(uden titel)"
|
||||
}
|
||||
|
||||
switch {
|
||||
case r.Start.Date != "":
|
||||
e.AllDay = true
|
||||
e.StartDay = r.Start.Date
|
||||
// Google's all-day end is the day *after* the last one, so a one-day event
|
||||
// ends tomorrow. Subtracting a day is what makes "covers this cell" a
|
||||
// comparison rather than a special case at every call site.
|
||||
e.EndDay = r.Start.Date
|
||||
if day, err := time.Parse(dateLayout, r.End.Date); err == nil {
|
||||
if last := day.AddDate(0, 0, -1); !last.Before(mustDay(r.Start.Date)) {
|
||||
e.EndDay = last.Format(dateLayout)
|
||||
}
|
||||
}
|
||||
case r.Start.DateTime != "":
|
||||
e.StartsAt = r.Start.DateTime
|
||||
e.EndsAt = r.End.DateTime
|
||||
// The day is the date part of the timestamp Google returned, which carries
|
||||
// the event's own offset. Parsing it into a time.Time and formatting it
|
||||
// back would ask *this process* what day it is — and a container running in
|
||||
// UTC would then move a 23:30 meeting in Copenhagen to the day before.
|
||||
e.StartDay = dayOf(r.Start.DateTime)
|
||||
e.EndDay = dayOf(r.End.DateTime)
|
||||
if e.EndDay == "" {
|
||||
e.EndDay = e.StartDay
|
||||
}
|
||||
// An event ending at midnight ends the evening before, as far as a grid is
|
||||
// concerned. Without this, every meeting that runs to 24:00 puts a chip in
|
||||
// the following day as well.
|
||||
if e.EndDay > e.StartDay && strings.Contains(r.End.DateTime, "T00:00:00") {
|
||||
if day, err := time.Parse(dateLayout, e.EndDay); err == nil {
|
||||
e.EndDay = day.AddDate(0, 0, -1).Format(dateLayout)
|
||||
}
|
||||
}
|
||||
default:
|
||||
return Event{}, false
|
||||
}
|
||||
if e.EndDay < e.StartDay {
|
||||
e.EndDay = e.StartDay
|
||||
}
|
||||
return e, true
|
||||
}
|
||||
|
||||
// dayOf takes the date out of an RFC 3339 timestamp without interpreting it.
|
||||
func dayOf(stamp string) string {
|
||||
if len(stamp) < len(dateLayout) {
|
||||
return ""
|
||||
}
|
||||
day := stamp[:len(dateLayout)]
|
||||
if _, err := time.Parse(dateLayout, day); err != nil {
|
||||
return ""
|
||||
}
|
||||
return day
|
||||
}
|
||||
|
||||
func mustDay(s string) time.Time {
|
||||
day, _ := time.Parse(dateLayout, s)
|
||||
return day
|
||||
}
|
||||
|
||||
func (c *Client) get(ctx context.Context, url string, out any) error {
|
||||
return google.Get(ctx, c.http, c.accessToken, url, out)
|
||||
}
|
||||
@@ -0,0 +1,255 @@
|
||||
package gcal
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// fakeGoogle answers the two calls this package makes, out of whatever the test
|
||||
// hands it. There is no other way to exercise this: the real Calendar needs an
|
||||
// account, a registered client and somebody's consent, and none of the three can be
|
||||
// had inside a test — so the shapes Google returns are written down here and the
|
||||
// parsing is checked against them.
|
||||
func fakeGoogle(t *testing.T, calendarList any, events map[string]any) *Client {
|
||||
t.Helper()
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
switch {
|
||||
case strings.HasPrefix(r.URL.Path, "/users/me/calendarList"):
|
||||
_ = json.NewEncoder(w).Encode(calendarList)
|
||||
case strings.HasPrefix(r.URL.Path, "/calendars/"):
|
||||
id := strings.TrimSuffix(strings.TrimPrefix(r.URL.Path, "/calendars/"), "/events")
|
||||
body, ok := events[id]
|
||||
if !ok {
|
||||
http.Error(w, `{"error":"not found"}`, http.StatusNotFound)
|
||||
return
|
||||
}
|
||||
_ = json.NewEncoder(w).Encode(body)
|
||||
default:
|
||||
http.Error(w, "unexpected path "+r.URL.Path, http.StatusNotFound)
|
||||
}
|
||||
}))
|
||||
t.Cleanup(srv.Close)
|
||||
return NewClient("token").At(srv.URL)
|
||||
}
|
||||
|
||||
func TestCalendarsPrefersTheNameThePersonGaveIt(t *testing.T) {
|
||||
client := fakeGoogle(t, map[string]any{
|
||||
"items": []map[string]any{
|
||||
{"id": "primary", "summary": "[email protected]", "backgroundColor": "#3f51b5",
|
||||
"timeZone": "Europe/Copenhagen", "primary": true, "accessRole": "owner"},
|
||||
{"id": "fam", "summary": "Family", "summaryOverride": "Familien",
|
||||
"backgroundColor": "#0b8043", "accessRole": "reader"},
|
||||
// A calendar removed from the list still arrives, flagged.
|
||||
{"id": "gone", "summary": "Væk", "deleted": true, "accessRole": "reader"},
|
||||
},
|
||||
}, nil)
|
||||
|
||||
got, err := client.Calendars(t.Context())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(got) != 2 {
|
||||
t.Fatalf("got %d calendars, want 2 — a deleted one is not one you can show", len(got))
|
||||
}
|
||||
if got[1].Name != "Familien" {
|
||||
t.Errorf("name = %q; summaryOverride is the name the person will look for", got[1].Name)
|
||||
}
|
||||
if !got[0].Primary || !got[0].Owner {
|
||||
t.Errorf("the primary calendar came back as %+v", got[0])
|
||||
}
|
||||
if got[1].Owner {
|
||||
t.Error("a reader was reported as an owner; nothing may offer to write to it")
|
||||
}
|
||||
if got[0].Colour != "#3f51b5" {
|
||||
t.Errorf("colour = %q; two calendars in one grid are told apart by it", got[0].Colour)
|
||||
}
|
||||
}
|
||||
|
||||
// The three shapes an event arrives in, and the one shape a grid can read.
|
||||
func TestEventsBecomeDaysAGridCanAskAbout(t *testing.T) {
|
||||
client := fakeGoogle(t, nil, map[string]any{
|
||||
"primary": map[string]any{
|
||||
"items": []map[string]any{
|
||||
{
|
||||
"id": "timed", "summary": "Bestyrelsesmøde", "htmlLink": "https://cal/1",
|
||||
"start": map[string]string{"dateTime": "2026-08-20T14:00:00+02:00"},
|
||||
"end": map[string]string{"dateTime": "2026-08-20T15:30:00+02:00"},
|
||||
},
|
||||
{
|
||||
// All-day, one day. Google's end is the day after.
|
||||
"id": "allday", "summary": "Fridag",
|
||||
"start": map[string]string{"date": "2026-08-21"},
|
||||
"end": map[string]string{"date": "2026-08-22"},
|
||||
},
|
||||
{
|
||||
// All-day, three days.
|
||||
"id": "trip", "summary": "Aarhus",
|
||||
"start": map[string]string{"date": "2026-08-24"},
|
||||
"end": map[string]string{"date": "2026-08-27"},
|
||||
},
|
||||
{
|
||||
// Runs to midnight, which is the evening before as far as a
|
||||
// grid is concerned.
|
||||
"id": "late", "summary": "Nattevagt",
|
||||
"start": map[string]string{"dateTime": "2026-08-22T20:00:00+02:00"},
|
||||
"end": map[string]string{"dateTime": "2026-08-23T00:00:00+02:00"},
|
||||
},
|
||||
{
|
||||
// A cancelled occurrence of a recurring event.
|
||||
"id": "off", "status": "cancelled", "summary": "Aflyst",
|
||||
"start": map[string]string{"date": "2026-08-25"},
|
||||
"end": map[string]string{"date": "2026-08-26"},
|
||||
},
|
||||
{
|
||||
// Neither a date nor a dateTime: belongs on no day at all.
|
||||
"id": "nowhere", "summary": "Uden tid",
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
got, err := client.Events(t.Context(), "primary", "2026-08-01", "2026-08-31")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
byID := map[string]Event{}
|
||||
for _, e := range got {
|
||||
byID[e.ID] = e
|
||||
}
|
||||
if len(got) != 4 {
|
||||
t.Fatalf("got %d events, want 4: %+v", len(got), got)
|
||||
}
|
||||
if _, ok := byID["off"]; ok {
|
||||
t.Error("a cancelled occurrence was kept; it is an event that is not happening")
|
||||
}
|
||||
if _, ok := byID["nowhere"]; ok {
|
||||
t.Error("an event with no time was kept; there is no cell to draw it in")
|
||||
}
|
||||
|
||||
timed := byID["timed"]
|
||||
if timed.AllDay || timed.StartDay != "2026-08-20" || timed.EndDay != "2026-08-20" {
|
||||
t.Errorf("timed = %+v", timed)
|
||||
}
|
||||
if timed.StartsAt != "2026-08-20T14:00:00+02:00" {
|
||||
t.Errorf("the instant was rewritten: %q", timed.StartsAt)
|
||||
}
|
||||
|
||||
allday := byID["allday"]
|
||||
if !allday.AllDay || allday.StartDay != "2026-08-21" || allday.EndDay != "2026-08-21" {
|
||||
t.Errorf("allday = %+v; Google's end date is exclusive and has to be corrected here",
|
||||
allday)
|
||||
}
|
||||
if allday.StartsAt != "" {
|
||||
t.Errorf("an all-day event was given an instant: %q — it has a day, not a moment",
|
||||
allday.StartsAt)
|
||||
}
|
||||
|
||||
if trip := byID["trip"]; trip.StartDay != "2026-08-24" || trip.EndDay != "2026-08-26" {
|
||||
t.Errorf("trip = %+v, want 24th to 26th inclusive", trip)
|
||||
}
|
||||
if late := byID["late"]; late.EndDay != "2026-08-22" {
|
||||
t.Errorf("late ends on %q; an event running to midnight ends the evening before, "+
|
||||
"or every one of them puts a chip in the following day", late.EndDay)
|
||||
}
|
||||
}
|
||||
|
||||
// An event with no title has to say so. A blank chip in a grid reads as a
|
||||
// rendering fault rather than as an empty title.
|
||||
func TestAnEventWithNoTitleStillSaysSomething(t *testing.T) {
|
||||
client := fakeGoogle(t, nil, map[string]any{
|
||||
"c": map[string]any{"items": []map[string]any{{
|
||||
"id": "x",
|
||||
"start": map[string]string{"date": "2026-08-20"},
|
||||
"end": map[string]string{"date": "2026-08-21"},
|
||||
}}},
|
||||
})
|
||||
got, err := client.Events(t.Context(), "c", "2026-08-01", "2026-08-31")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(got) != 1 || got[0].Summary == "" {
|
||||
t.Fatalf("got %+v", got)
|
||||
}
|
||||
}
|
||||
|
||||
// The window is asked for with a day of slack at each end, because an event late
|
||||
// in the evening in Copenhagen is the next day in UTC. The slack must not leak
|
||||
// out: an event on the day before the window is not in the window.
|
||||
func TestTheSlackAtTheEdgesIsThrownAwayAgain(t *testing.T) {
|
||||
client := fakeGoogle(t, nil, map[string]any{
|
||||
"c": map[string]any{"items": []map[string]any{
|
||||
{"id": "before", "summary": "For tidligt",
|
||||
"start": map[string]string{"date": "2026-08-09"},
|
||||
"end": map[string]string{"date": "2026-08-10"}},
|
||||
{"id": "inside", "summary": "Med",
|
||||
"start": map[string]string{"date": "2026-08-10"},
|
||||
"end": map[string]string{"date": "2026-08-11"}},
|
||||
{"id": "after", "summary": "For sent",
|
||||
"start": map[string]string{"date": "2026-08-21"},
|
||||
"end": map[string]string{"date": "2026-08-22"}},
|
||||
}},
|
||||
})
|
||||
got, err := client.Events(t.Context(), "c", "2026-08-10", "2026-08-20")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(got) != 1 || got[0].ID != "inside" {
|
||||
t.Fatalf("got %+v, want only the one inside the window", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A page token has to be followed, or a busy calendar silently shows its first
|
||||
// page and nothing else — which is a calendar that lies about a day being clear.
|
||||
func TestEveryPageIsRead(t *testing.T) {
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
body := map[string]any{
|
||||
"items": []map[string]any{{"id": "a", "summary": "Første",
|
||||
"start": map[string]string{"date": "2026-08-11"},
|
||||
"end": map[string]string{"date": "2026-08-12"}}},
|
||||
"nextPageToken": "more",
|
||||
}
|
||||
if r.URL.Query().Get("pageToken") == "more" {
|
||||
body = map[string]any{"items": []map[string]any{{"id": "b", "summary": "Anden",
|
||||
"start": map[string]string{"date": "2026-08-12"},
|
||||
"end": map[string]string{"date": "2026-08-13"}}}}
|
||||
}
|
||||
_ = json.NewEncoder(w).Encode(body)
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
got, err := NewClient("t").At(srv.URL).Events(t.Context(), "c", "2026-08-01", "2026-08-31")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(got) != 2 {
|
||||
t.Fatalf("got %d events, want both pages: %+v", len(got), got)
|
||||
}
|
||||
}
|
||||
|
||||
// A 401 is not a failure to retry; it is a connection that has ended. Telling the
|
||||
// two apart is what stops a revoked account being polled every ten minutes for ever.
|
||||
func TestARefusedTokenIsItsOwnError(t *testing.T) {
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
http.Error(w, `{"error":"invalid_credentials"}`, http.StatusUnauthorized)
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
_, err := NewClient("spent").At(srv.URL).Calendars(t.Context())
|
||||
if err != ErrUnauthorized {
|
||||
t.Fatalf("err = %v, want ErrUnauthorized", err)
|
||||
}
|
||||
}
|
||||
|
||||
// Read-only, and only the calendar. The consent screen is what somebody agrees to,
|
||||
// and a scope that quietly grew is a promise quietly broken.
|
||||
func TestTheScopeIsReadOnly(t *testing.T) {
|
||||
if !strings.HasSuffix(Scope, "calendar.readonly") {
|
||||
t.Errorf("scope = %q; nothing here writes a calendar", Scope)
|
||||
}
|
||||
}
|
||||
@@ -1,5 +1,5 @@
|
||||
// Package gmail implements the OAuth2 authorisation-code flow with PKCE against
|
||||
// Google, and the small slice of the Gmail API verdande needs.
|
||||
// Package gmail is the small slice of the Gmail API verdande needs, and the scope
|
||||
// it asks for.
|
||||
//
|
||||
// The connection is one-way on purpose: a starred or labelled message becomes a
|
||||
// task, and nothing verdande does afterwards touches the mailbox. Unstarring the
|
||||
@@ -9,200 +9,51 @@
|
||||
//
|
||||
// Scope is gmail.readonly. verdande never needs to send, modify or delete, and
|
||||
// asking for less is what makes the consent screen something a person can agree to.
|
||||
//
|
||||
// The OAuth flow itself moved to internal/google when Calendar needed the same one.
|
||||
// The names below are aliases so that every call site — and this package's own
|
||||
// tests — still read as Gmail's, which is what they are: one registration, one
|
||||
// flow, one scope each.
|
||||
package gmail
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/kristianwind/verdande/internal/google"
|
||||
)
|
||||
|
||||
const (
|
||||
authEndpoint = "https://accounts.google.com/o/oauth2/v2/auth"
|
||||
defaultTokenEndpoint = "https://oauth2.googleapis.com/token"
|
||||
defaultAPIBase = "https://gmail.googleapis.com/gmail/v1"
|
||||
// Read-only, and only mail. Not profile, not contacts, not send.
|
||||
const Scope = "https://www.googleapis.com/auth/gmail.readonly"
|
||||
|
||||
// Read-only, and only mail. Not profile, not contacts, not send.
|
||||
Scope = "https://www.googleapis.com/auth/gmail.readonly"
|
||||
const defaultAPIBase = "https://gmail.googleapis.com/gmail/v1"
|
||||
|
||||
// The OAuth flow, which is Google's rather than Gmail's.
|
||||
type (
|
||||
Config = google.Config
|
||||
PKCE = google.PKCE
|
||||
Token = google.Token
|
||||
)
|
||||
|
||||
// Endpoints is where Google is.
|
||||
//
|
||||
// A field rather than a constant so a test can point it at a server it controls.
|
||||
// There is no other way to exercise what happens when Gmail is slow or refuses —
|
||||
// and "the sync comes back before whatever sits in front of this server gives up
|
||||
// on it" is exactly the kind of promise worth a test and impossible to make
|
||||
// against the real thing.
|
||||
//
|
||||
// The zero value is Google, so nothing outside a test has to know it exists.
|
||||
type Endpoints struct {
|
||||
Token string
|
||||
API string
|
||||
}
|
||||
|
||||
func (e Endpoints) token() string {
|
||||
if e.Token != "" {
|
||||
return e.Token
|
||||
}
|
||||
return defaultTokenEndpoint
|
||||
}
|
||||
|
||||
func (e Endpoints) api() string {
|
||||
if e.API != "" {
|
||||
return e.API
|
||||
}
|
||||
return defaultAPIBase
|
||||
}
|
||||
|
||||
// Config is the OAuth client the operator registered in Google Cloud. It belongs to
|
||||
// the instance rather than to a person: one registration, any number of users.
|
||||
type Config struct {
|
||||
ClientID string
|
||||
ClientSecret string
|
||||
RedirectURL string
|
||||
// Endpoints is Google unless a test says otherwise.
|
||||
Endpoints Endpoints
|
||||
}
|
||||
|
||||
func (c Config) Configured() bool {
|
||||
return c.ClientID != "" && c.ClientSecret != "" && c.RedirectURL != ""
|
||||
}
|
||||
|
||||
// PKCE is the proof-key pair for one authorisation attempt.
|
||||
//
|
||||
// PKCE on a server-side client is not strictly required — the client secret already
|
||||
// authenticates the token exchange — but it costs nothing and closes the window in
|
||||
// which an authorisation code intercepted from the redirect could be replayed
|
||||
// before verdande gets to it.
|
||||
type PKCE struct {
|
||||
Verifier string
|
||||
Challenge string
|
||||
State string
|
||||
}
|
||||
|
||||
func NewPKCE() (PKCE, error) {
|
||||
verifier, err := randomString(64)
|
||||
if err != nil {
|
||||
return PKCE{}, err
|
||||
}
|
||||
state, err := randomString(32)
|
||||
if err != nil {
|
||||
return PKCE{}, err
|
||||
}
|
||||
sum := sha256.Sum256([]byte(verifier))
|
||||
return PKCE{
|
||||
Verifier: verifier,
|
||||
Challenge: base64.RawURLEncoding.EncodeToString(sum[:]),
|
||||
State: state,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// AuthURL is where the browser is sent to ask the person for consent.
|
||||
func (c Config) AuthURL(p PKCE) string {
|
||||
v := url.Values{}
|
||||
v.Set("client_id", c.ClientID)
|
||||
v.Set("redirect_uri", c.RedirectURL)
|
||||
v.Set("response_type", "code")
|
||||
v.Set("scope", Scope)
|
||||
v.Set("state", p.State)
|
||||
v.Set("code_challenge", p.Challenge)
|
||||
v.Set("code_challenge_method", "S256")
|
||||
// Without both of these Google returns no refresh token on a re-authorisation,
|
||||
// and the connection silently stops working an hour later.
|
||||
v.Set("access_type", "offline")
|
||||
v.Set("prompt", "consent")
|
||||
return authEndpoint + "?" + v.Encode()
|
||||
}
|
||||
|
||||
type Token struct {
|
||||
AccessToken string
|
||||
RefreshToken string
|
||||
ExpiresAt time.Time
|
||||
}
|
||||
|
||||
// Exchange trades the authorisation code for tokens.
|
||||
func (c Config) Exchange(ctx context.Context, code string, p PKCE) (Token, error) {
|
||||
form := url.Values{}
|
||||
form.Set("code", code)
|
||||
form.Set("client_id", c.ClientID)
|
||||
form.Set("client_secret", c.ClientSecret)
|
||||
form.Set("redirect_uri", c.RedirectURL)
|
||||
form.Set("grant_type", "authorization_code")
|
||||
form.Set("code_verifier", p.Verifier)
|
||||
|
||||
return c.token(ctx, form)
|
||||
}
|
||||
|
||||
// Refresh gets a fresh access token. Google does not return a new refresh token
|
||||
// here, so the caller keeps the one it has.
|
||||
func (c Config) Refresh(ctx context.Context, refreshToken string) (Token, error) {
|
||||
form := url.Values{}
|
||||
form.Set("refresh_token", refreshToken)
|
||||
form.Set("client_id", c.ClientID)
|
||||
form.Set("client_secret", c.ClientSecret)
|
||||
form.Set("grant_type", "refresh_token")
|
||||
|
||||
token, err := c.token(ctx, form)
|
||||
if err != nil {
|
||||
return Token{}, err
|
||||
}
|
||||
token.RefreshToken = refreshToken
|
||||
return token, nil
|
||||
}
|
||||
|
||||
func (c Config) token(ctx context.Context, form url.Values) (Token, error) {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.Endpoints.token(),
|
||||
strings.NewReader(form.Encode()))
|
||||
if err != nil {
|
||||
return Token{}, err
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
|
||||
|
||||
resp, err := (&http.Client{Timeout: 30 * time.Second}).Do(req)
|
||||
if err != nil {
|
||||
return Token{}, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
var parsed struct {
|
||||
AccessToken string `json:"access_token"`
|
||||
RefreshToken string `json:"refresh_token"`
|
||||
ExpiresIn int `json:"expires_in"`
|
||||
Error string `json:"error"`
|
||||
ErrorDescription string `json:"error_description"`
|
||||
}
|
||||
if err := json.NewDecoder(resp.Body).Decode(&parsed); err != nil {
|
||||
return Token{}, fmt.Errorf("gmail: reading token response: %w", err)
|
||||
}
|
||||
if parsed.Error != "" {
|
||||
return Token{}, fmt.Errorf("gmail: %s: %s", parsed.Error, parsed.ErrorDescription)
|
||||
}
|
||||
if resp.StatusCode != http.StatusOK || parsed.AccessToken == "" {
|
||||
return Token{}, fmt.Errorf("gmail: token exchange failed with %s", resp.Status)
|
||||
}
|
||||
|
||||
return Token{
|
||||
AccessToken: parsed.AccessToken,
|
||||
RefreshToken: parsed.RefreshToken,
|
||||
// A minute of slack, so a token that is about to expire is refreshed
|
||||
// rather than used and rejected.
|
||||
ExpiresAt: time.Now().Add(time.Duration(parsed.ExpiresIn-60) * time.Second),
|
||||
}, nil
|
||||
}
|
||||
var (
|
||||
NewPKCE = google.NewPKCE
|
||||
ErrUnauthorized = google.ErrUnauthorized
|
||||
)
|
||||
|
||||
// --- the API ---------------------------------------------------------------------
|
||||
|
||||
type Client struct {
|
||||
accessToken string
|
||||
endpoints Endpoints
|
||||
http *http.Client
|
||||
// base is where the Gmail API is. A field rather than a constant so a test can
|
||||
// point it at a server it controls — and "the sync comes back before whatever
|
||||
// sits in front of this server gives up on it" is exactly the kind of promise
|
||||
// worth a test and impossible to make against the real thing. Empty is Google.
|
||||
base string
|
||||
http *http.Client
|
||||
}
|
||||
|
||||
func NewClient(accessToken string) *Client {
|
||||
@@ -210,11 +61,18 @@ func NewClient(accessToken string) *Client {
|
||||
}
|
||||
|
||||
// At points the client at another server. Tests only; the zero value is Google.
|
||||
func (c *Client) At(e Endpoints) *Client {
|
||||
c.endpoints = e
|
||||
func (c *Client) At(base string) *Client {
|
||||
c.base = base
|
||||
return c
|
||||
}
|
||||
|
||||
func (c *Client) api() string {
|
||||
if c.base != "" {
|
||||
return c.base
|
||||
}
|
||||
return defaultAPIBase
|
||||
}
|
||||
|
||||
// Message is the part of a Gmail message a task is made from.
|
||||
type Message struct {
|
||||
ID string
|
||||
@@ -233,7 +91,7 @@ func (c *Client) Profile(ctx context.Context) (string, error) {
|
||||
var parsed struct {
|
||||
EmailAddress string `json:"emailAddress"`
|
||||
}
|
||||
if err := c.get(ctx, c.endpoints.api()+"/users/me/profile", &parsed); err != nil {
|
||||
if err := c.get(ctx, c.api()+"/users/me/profile", &parsed); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return parsed.EmailAddress, nil
|
||||
@@ -257,7 +115,7 @@ func (c *Client) List(ctx context.Context, query string, max int) ([]string, err
|
||||
ID string `json:"id"`
|
||||
} `json:"messages"`
|
||||
}
|
||||
if err := c.get(ctx, c.endpoints.api()+"/users/me/messages?"+v.Encode(), &parsed); err != nil {
|
||||
if err := c.get(ctx, c.api()+"/users/me/messages?"+v.Encode(), &parsed); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
@@ -288,7 +146,7 @@ func (c *Client) Get(ctx context.Context, id string) (Message, error) {
|
||||
} `json:"headers"`
|
||||
} `json:"payload"`
|
||||
}
|
||||
if err := c.get(ctx, c.endpoints.api()+"/users/me/messages/"+url.PathEscape(id)+"?"+v.Encode(), &parsed); err != nil {
|
||||
if err := c.get(ctx, c.api()+"/users/me/messages/"+url.PathEscape(id)+"?"+v.Encode(), &parsed); err != nil {
|
||||
return Message{}, err
|
||||
}
|
||||
|
||||
@@ -308,31 +166,9 @@ func (c *Client) Get(ctx context.Context, id string) (Message, error) {
|
||||
}
|
||||
|
||||
func (c *Client) get(ctx context.Context, url string, out any) error {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("Authorization", "Bearer "+c.accessToken)
|
||||
|
||||
resp, err := c.http.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode == http.StatusUnauthorized {
|
||||
return ErrUnauthorized
|
||||
}
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return fmt.Errorf("gmail: %s", resp.Status)
|
||||
}
|
||||
return json.NewDecoder(resp.Body).Decode(out)
|
||||
return google.Get(ctx, c.http, c.accessToken, url, out)
|
||||
}
|
||||
|
||||
// ErrUnauthorized means the tokens are no longer good — the person revoked access,
|
||||
// or changed their password. The caller disconnects rather than retrying.
|
||||
var ErrUnauthorized = fmt.Errorf("gmail: authorisation was refused")
|
||||
|
||||
// SenderName pulls "Anders Jensen" out of "Anders Jensen <[email protected]>",
|
||||
// falling back to the address when there is no display name.
|
||||
func SenderName(from string) string {
|
||||
@@ -375,11 +211,3 @@ func Query(trigger, label string) string {
|
||||
terms = append(terms, "newer_than:30d")
|
||||
return strings.Join(terms, " ")
|
||||
}
|
||||
|
||||
func randomString(n int) (string, error) {
|
||||
b := make([]byte, n)
|
||||
if _, err := rand.Read(b); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return base64.RawURLEncoding.EncodeToString(b)[:n], nil
|
||||
}
|
||||
|
||||
@@ -1,113 +1,19 @@
|
||||
package gmail
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/base64"
|
||||
"net/url"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The challenge has to be the SHA-256 of the verifier, base64url without padding.
|
||||
// Getting this wrong means Google refuses every exchange with a message that does
|
||||
// not say which end is at fault.
|
||||
func TestPKCEChallengeMatchesItsVerifier(t *testing.T) {
|
||||
p, err := NewPKCE()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
sum := sha256.Sum256([]byte(p.Verifier))
|
||||
want := base64.RawURLEncoding.EncodeToString(sum[:])
|
||||
if p.Challenge != want {
|
||||
t.Errorf("challenge = %q, want %q", p.Challenge, want)
|
||||
}
|
||||
|
||||
// RFC 7636 requires 43–128 characters, and no padding.
|
||||
if n := len(p.Verifier); n < 43 || n > 128 {
|
||||
t.Errorf("verifier is %d characters, outside the 43–128 the spec allows", n)
|
||||
}
|
||||
if strings.ContainsAny(p.Verifier, "=+/") {
|
||||
t.Errorf("verifier %q contains characters that are not URL-safe", p.Verifier)
|
||||
}
|
||||
if p.State == "" {
|
||||
t.Error("no state was generated")
|
||||
}
|
||||
|
||||
// Two attempts must not share anything.
|
||||
other, err := NewPKCE()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if other.Verifier == p.Verifier || other.State == p.State {
|
||||
t.Error("two authorisation attempts produced the same secrets")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAuthURL(t *testing.T) {
|
||||
cfg := Config{
|
||||
ClientID: "client-123",
|
||||
ClientSecret: "secret",
|
||||
RedirectURL: "https://todo.example.dk/oauth/gmail/callback",
|
||||
}
|
||||
p, err := NewPKCE()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
raw := cfg.AuthURL(p)
|
||||
parsed, err := url.Parse(raw)
|
||||
if err != nil {
|
||||
t.Fatalf("the URL does not parse: %v", err)
|
||||
}
|
||||
q := parsed.Query()
|
||||
|
||||
checks := map[string]string{
|
||||
"client_id": cfg.ClientID,
|
||||
"redirect_uri": cfg.RedirectURL,
|
||||
"response_type": "code",
|
||||
"scope": Scope,
|
||||
"code_challenge": p.Challenge,
|
||||
"code_challenge_method": "S256",
|
||||
"state": p.State,
|
||||
// Without both of these Google returns no refresh token on a
|
||||
// re-authorisation, and the connection dies silently an hour later.
|
||||
"access_type": "offline",
|
||||
"prompt": "consent",
|
||||
}
|
||||
for key, want := range checks {
|
||||
if got := q.Get(key); got != want {
|
||||
t.Errorf("%s = %q, want %q", key, got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// The verifier itself must never leave the server.
|
||||
if strings.Contains(raw, p.Verifier) {
|
||||
t.Error("the code verifier was put in the authorisation URL")
|
||||
}
|
||||
// Read-only, and only mail.
|
||||
// The scope is the one thing verdande chooses about the connection, and it is the
|
||||
// one the person reads on the consent screen. The flow itself is exercised in
|
||||
// internal/google; what belongs here is that Gmail still asks for read-only mail.
|
||||
func TestTheScopeIsReadOnlyMail(t *testing.T) {
|
||||
if !strings.HasSuffix(Scope, "gmail.readonly") {
|
||||
t.Errorf("scope = %q; verdande never needs to send or modify", Scope)
|
||||
}
|
||||
}
|
||||
|
||||
func TestConfigured(t *testing.T) {
|
||||
full := Config{ClientID: "a", ClientSecret: "b", RedirectURL: "c"}
|
||||
if !full.Configured() {
|
||||
t.Error("a complete config is not reported as configured")
|
||||
}
|
||||
for _, partial := range []Config{
|
||||
{},
|
||||
{ClientID: "a"},
|
||||
{ClientID: "a", ClientSecret: "b"},
|
||||
{ClientSecret: "b", RedirectURL: "c"},
|
||||
} {
|
||||
if partial.Configured() {
|
||||
t.Errorf("%+v was reported as configured", partial)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestQuery(t *testing.T) {
|
||||
cases := []struct {
|
||||
trigger, label string
|
||||
|
||||
@@ -0,0 +1,224 @@
|
||||
// Package google implements the OAuth2 authorisation-code flow with PKCE against
|
||||
// Google.
|
||||
//
|
||||
// It was written inside internal/gmail, because Gmail was the only thing that
|
||||
// needed it. Calendar needs the same flow — the same endpoints, the same PKCE, the
|
||||
// same refresh — and the only thing that differs between them is one string: the
|
||||
// scope. A second copy would be a second place for "access_type=offline" to be
|
||||
// forgotten, which is the mistake that makes a connection die silently an hour
|
||||
// after it was made.
|
||||
//
|
||||
// What is *not* here is the API calls. Those differ completely, so gmail and gcal
|
||||
// keep their own clients and share only the way in.
|
||||
package google
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
const (
|
||||
authEndpoint = "https://accounts.google.com/o/oauth2/v2/auth"
|
||||
defaultTokenEndpoint = "https://oauth2.googleapis.com/token"
|
||||
)
|
||||
|
||||
// Config is the OAuth client the operator registered in Google Cloud, plus the
|
||||
// scope one feature is asking for.
|
||||
//
|
||||
// The client belongs to the instance rather than to a person: one registration,
|
||||
// any number of users. The scope belongs to the feature: Gmail and Calendar sign
|
||||
// in through the same registration and ask for different things, and Google issues
|
||||
// a refresh token per authorisation — so connecting one does not disturb the other.
|
||||
type Config struct {
|
||||
ClientID string
|
||||
ClientSecret string
|
||||
RedirectURL string
|
||||
Scope string
|
||||
|
||||
// TokenURL is where Google is. A field rather than a constant so a test can
|
||||
// point it at a server it controls: there is no other way to exercise what
|
||||
// happens when the token endpoint is slow or refuses. Empty is Google, so
|
||||
// nothing outside a test has to know it exists.
|
||||
TokenURL string
|
||||
}
|
||||
|
||||
func (c Config) Configured() bool {
|
||||
return c.ClientID != "" && c.ClientSecret != "" && c.RedirectURL != ""
|
||||
}
|
||||
|
||||
func (c Config) tokenURL() string {
|
||||
if c.TokenURL != "" {
|
||||
return c.TokenURL
|
||||
}
|
||||
return defaultTokenEndpoint
|
||||
}
|
||||
|
||||
// PKCE is the proof-key pair for one authorisation attempt.
|
||||
//
|
||||
// PKCE on a server-side client is not strictly required — the client secret already
|
||||
// authenticates the token exchange — but it costs nothing and closes the window in
|
||||
// which an authorisation code intercepted from the redirect could be replayed
|
||||
// before verdande gets to it.
|
||||
type PKCE struct {
|
||||
Verifier string
|
||||
Challenge string
|
||||
State string
|
||||
}
|
||||
|
||||
func NewPKCE() (PKCE, error) {
|
||||
verifier, err := randomString(64)
|
||||
if err != nil {
|
||||
return PKCE{}, err
|
||||
}
|
||||
state, err := randomString(32)
|
||||
if err != nil {
|
||||
return PKCE{}, err
|
||||
}
|
||||
sum := sha256.Sum256([]byte(verifier))
|
||||
return PKCE{
|
||||
Verifier: verifier,
|
||||
Challenge: base64.RawURLEncoding.EncodeToString(sum[:]),
|
||||
State: state,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// AuthURL is where the browser is sent to ask the person for consent.
|
||||
func (c Config) AuthURL(p PKCE) string {
|
||||
v := url.Values{}
|
||||
v.Set("client_id", c.ClientID)
|
||||
v.Set("redirect_uri", c.RedirectURL)
|
||||
v.Set("response_type", "code")
|
||||
v.Set("scope", c.Scope)
|
||||
v.Set("state", p.State)
|
||||
v.Set("code_challenge", p.Challenge)
|
||||
v.Set("code_challenge_method", "S256")
|
||||
// Without both of these Google returns no refresh token on a re-authorisation,
|
||||
// and the connection silently stops working an hour later.
|
||||
v.Set("access_type", "offline")
|
||||
v.Set("prompt", "consent")
|
||||
return authEndpoint + "?" + v.Encode()
|
||||
}
|
||||
|
||||
type Token struct {
|
||||
AccessToken string
|
||||
RefreshToken string
|
||||
ExpiresAt time.Time
|
||||
}
|
||||
|
||||
// Exchange trades the authorisation code for tokens.
|
||||
func (c Config) Exchange(ctx context.Context, code string, p PKCE) (Token, error) {
|
||||
form := url.Values{}
|
||||
form.Set("code", code)
|
||||
form.Set("client_id", c.ClientID)
|
||||
form.Set("client_secret", c.ClientSecret)
|
||||
form.Set("redirect_uri", c.RedirectURL)
|
||||
form.Set("grant_type", "authorization_code")
|
||||
form.Set("code_verifier", p.Verifier)
|
||||
|
||||
return c.token(ctx, form)
|
||||
}
|
||||
|
||||
// Refresh gets a fresh access token. Google does not return a new refresh token
|
||||
// here, so the caller keeps the one it has.
|
||||
func (c Config) Refresh(ctx context.Context, refreshToken string) (Token, error) {
|
||||
form := url.Values{}
|
||||
form.Set("refresh_token", refreshToken)
|
||||
form.Set("client_id", c.ClientID)
|
||||
form.Set("client_secret", c.ClientSecret)
|
||||
form.Set("grant_type", "refresh_token")
|
||||
|
||||
token, err := c.token(ctx, form)
|
||||
if err != nil {
|
||||
return Token{}, err
|
||||
}
|
||||
token.RefreshToken = refreshToken
|
||||
return token, nil
|
||||
}
|
||||
|
||||
func (c Config) token(ctx context.Context, form url.Values) (Token, error) {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.tokenURL(),
|
||||
strings.NewReader(form.Encode()))
|
||||
if err != nil {
|
||||
return Token{}, err
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
|
||||
|
||||
resp, err := (&http.Client{Timeout: 30 * time.Second}).Do(req)
|
||||
if err != nil {
|
||||
return Token{}, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
var parsed struct {
|
||||
AccessToken string `json:"access_token"`
|
||||
RefreshToken string `json:"refresh_token"`
|
||||
ExpiresIn int `json:"expires_in"`
|
||||
Error string `json:"error"`
|
||||
ErrorDescription string `json:"error_description"`
|
||||
}
|
||||
if err := json.NewDecoder(resp.Body).Decode(&parsed); err != nil {
|
||||
return Token{}, fmt.Errorf("google: reading token response: %w", err)
|
||||
}
|
||||
if parsed.Error != "" {
|
||||
return Token{}, fmt.Errorf("google: %s: %s", parsed.Error, parsed.ErrorDescription)
|
||||
}
|
||||
if resp.StatusCode != http.StatusOK || parsed.AccessToken == "" {
|
||||
return Token{}, fmt.Errorf("google: token exchange failed with %s", resp.Status)
|
||||
}
|
||||
|
||||
return Token{
|
||||
AccessToken: parsed.AccessToken,
|
||||
RefreshToken: parsed.RefreshToken,
|
||||
// A minute of slack, so a token that is about to expire is refreshed
|
||||
// rather than used and rejected.
|
||||
ExpiresAt: time.Now().Add(time.Duration(parsed.ExpiresIn-60) * time.Second),
|
||||
}, nil
|
||||
}
|
||||
|
||||
// ErrUnauthorized means the tokens are no longer good — the person revoked access,
|
||||
// or changed their password. The caller disconnects rather than retrying.
|
||||
var ErrUnauthorized = fmt.Errorf("google: authorisation was refused")
|
||||
|
||||
// Get performs an authorised GET and decodes the JSON body into out.
|
||||
//
|
||||
// Shared by the two API clients because the interesting part is the same in both:
|
||||
// a 401 is not a failure to retry, it is a connection that has ended, and telling
|
||||
// those apart is what stops a revoked account being polled every ten minutes for
|
||||
// ever.
|
||||
func Get(ctx context.Context, client *http.Client, accessToken, url string, out any) error {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("Authorization", "Bearer "+accessToken)
|
||||
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode == http.StatusUnauthorized {
|
||||
return ErrUnauthorized
|
||||
}
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return fmt.Errorf("google: %s", resp.Status)
|
||||
}
|
||||
return json.NewDecoder(resp.Body).Decode(out)
|
||||
}
|
||||
|
||||
func randomString(n int) (string, error) {
|
||||
b := make([]byte, n)
|
||||
if _, err := rand.Read(b); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return base64.RawURLEncoding.EncodeToString(b)[:n], nil
|
||||
}
|
||||
@@ -0,0 +1,127 @@
|
||||
package google
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/base64"
|
||||
"net/url"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The challenge has to be the SHA-256 of the verifier, base64url without padding.
|
||||
// Getting this wrong means Google refuses every exchange with a message that does
|
||||
// not say which end is at fault.
|
||||
func TestPKCEChallengeMatchesItsVerifier(t *testing.T) {
|
||||
p, err := NewPKCE()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
sum := sha256.Sum256([]byte(p.Verifier))
|
||||
want := base64.RawURLEncoding.EncodeToString(sum[:])
|
||||
if p.Challenge != want {
|
||||
t.Errorf("challenge = %q, want %q", p.Challenge, want)
|
||||
}
|
||||
|
||||
// RFC 7636 requires 43–128 characters, and no padding.
|
||||
if n := len(p.Verifier); n < 43 || n > 128 {
|
||||
t.Errorf("verifier is %d characters, outside the 43–128 the spec allows", n)
|
||||
}
|
||||
if strings.ContainsAny(p.Verifier, "=+/") {
|
||||
t.Errorf("verifier %q contains characters that are not URL-safe", p.Verifier)
|
||||
}
|
||||
if p.State == "" {
|
||||
t.Error("no state was generated")
|
||||
}
|
||||
|
||||
// Two attempts must not share anything.
|
||||
other, err := NewPKCE()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if other.Verifier == p.Verifier || other.State == p.State {
|
||||
t.Error("two authorisation attempts produced the same secrets")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAuthURL(t *testing.T) {
|
||||
cfg := Config{
|
||||
ClientID: "client-123",
|
||||
ClientSecret: "secret",
|
||||
RedirectURL: "https://todo.example.dk/oauth/gmail/callback",
|
||||
Scope: "https://www.googleapis.com/auth/gmail.readonly",
|
||||
}
|
||||
p, err := NewPKCE()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
raw := cfg.AuthURL(p)
|
||||
parsed, err := url.Parse(raw)
|
||||
if err != nil {
|
||||
t.Fatalf("the URL does not parse: %v", err)
|
||||
}
|
||||
q := parsed.Query()
|
||||
|
||||
checks := map[string]string{
|
||||
"client_id": cfg.ClientID,
|
||||
"redirect_uri": cfg.RedirectURL,
|
||||
"response_type": "code",
|
||||
"scope": cfg.Scope,
|
||||
"code_challenge": p.Challenge,
|
||||
"code_challenge_method": "S256",
|
||||
"state": p.State,
|
||||
// Without both of these Google returns no refresh token on a
|
||||
// re-authorisation, and the connection dies silently an hour later.
|
||||
"access_type": "offline",
|
||||
"prompt": "consent",
|
||||
}
|
||||
for key, want := range checks {
|
||||
if got := q.Get(key); got != want {
|
||||
t.Errorf("%s = %q, want %q", key, got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// The verifier itself must never leave the server.
|
||||
if strings.Contains(raw, p.Verifier) {
|
||||
t.Error("the code verifier was put in the authorisation URL")
|
||||
}
|
||||
}
|
||||
|
||||
// The scope is the one thing that differs between the two features signing in
|
||||
// through this, so it has to come out of the config rather than out of a constant
|
||||
// somebody could forget to change.
|
||||
func TestTheScopeComesFromTheConfig(t *testing.T) {
|
||||
p, err := NewPKCE()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
calendar := "https://www.googleapis.com/auth/calendar.readonly"
|
||||
raw := Config{ClientID: "a", Scope: calendar}.AuthURL(p)
|
||||
|
||||
parsed, err := url.Parse(raw)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := parsed.Query().Get("scope"); got != calendar {
|
||||
t.Errorf("scope = %q, want %q — asking for the wrong one is a consent screen "+
|
||||
"that grants the wrong thing", got, calendar)
|
||||
}
|
||||
}
|
||||
|
||||
func TestConfigured(t *testing.T) {
|
||||
full := Config{ClientID: "a", ClientSecret: "b", RedirectURL: "c"}
|
||||
if !full.Configured() {
|
||||
t.Error("a complete config is not reported as configured")
|
||||
}
|
||||
for _, partial := range []Config{
|
||||
{},
|
||||
{ClientID: "a"},
|
||||
{ClientID: "a", ClientSecret: "b"},
|
||||
{ClientSecret: "b", RedirectURL: "c"},
|
||||
} {
|
||||
if partial.Configured() {
|
||||
t.Errorf("%+v was reported as configured", partial)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -32,6 +32,7 @@ type userJSON struct {
|
||||
// about the work and not about the screen — the sidebar's width is the
|
||||
// opposite case and stays in localStorage.
|
||||
SidebarCollapsed []string `json:"sidebar_collapsed"`
|
||||
NavOrder []string `json:"nav_order"`
|
||||
CreatedAt string `json:"created_at"`
|
||||
}
|
||||
|
||||
@@ -42,10 +43,14 @@ func toUserJSON(u *store.User) userJSON {
|
||||
if collapsed == nil {
|
||||
collapsed = []string{}
|
||||
}
|
||||
order := u.NavOrder
|
||||
if order == nil {
|
||||
order = []string{}
|
||||
}
|
||||
return userJSON{
|
||||
ID: u.ID, Email: u.Email, Name: u.Name, AvatarColor: u.AvatarColor,
|
||||
Timezone: u.Timezone, Locale: u.Locale, IsAdmin: u.IsAdmin,
|
||||
TOTPEnabled: u.TOTPEnabled, SidebarCollapsed: collapsed,
|
||||
TOTPEnabled: u.TOTPEnabled, SidebarCollapsed: collapsed, NavOrder: order,
|
||||
CreatedAt: u.CreatedAt.Format(time.RFC3339),
|
||||
}
|
||||
}
|
||||
@@ -83,6 +88,30 @@ func (s *Server) handleSetSidebarSections(w http.ResponseWriter, r *http.Request
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
}
|
||||
|
||||
// handleSetNavOrder records the order of the fixed views in the sidebar.
|
||||
//
|
||||
// Its own route for the same reason as the folds: this writes on every drop, and
|
||||
// sharing the profile's path would make a reorder send a name and a timezone it
|
||||
// was never asked to change.
|
||||
func (s *Server) handleSetNavOrder(w http.ResponseWriter, r *http.Request) {
|
||||
var req struct {
|
||||
Order []string `json:"order"`
|
||||
}
|
||||
if err := decodeJSON(w, r, &req); err != nil {
|
||||
return
|
||||
}
|
||||
if len(req.Order) > 20 {
|
||||
writeError(w, http.StatusBadRequest, CodeBadRequest, "too many entries")
|
||||
return
|
||||
}
|
||||
user := userFrom(r.Context())
|
||||
if err := s.db.SetNavOrder(r.Context(), user.ID, req.Order); err != nil {
|
||||
s.storeError(w, r, "set nav order", err)
|
||||
return
|
||||
}
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
}
|
||||
|
||||
// --- sign in -----------------------------------------------------------------
|
||||
|
||||
type loginRequest struct {
|
||||
@@ -125,7 +154,21 @@ func (s *Server) handleLogin(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
}
|
||||
|
||||
s.loginLimiter.reset(clientIP(r))
|
||||
// The bucket is cleared only when the login is *finished*.
|
||||
//
|
||||
// It used to be cleared here, on the strength of the password alone — before
|
||||
// the second factor had been asked for at all. That handed somebody who
|
||||
// already had the password an unlimited supply of ten-guess batches: log in,
|
||||
// get a fresh bucket and a fresh pending session, spend ten guesses on the
|
||||
// code, log in again. At three valid codes in any moment and twenty requests a
|
||||
// second, that is even odds inside an afternoon — against the one control that
|
||||
// exists precisely for when the password is already gone.
|
||||
//
|
||||
// So an account with a second factor clears nothing yet. handleLoginTOTP does
|
||||
// it, once the code is right.
|
||||
if !user.TOTPEnabled {
|
||||
s.loginLimiter.reset(clientIP(r))
|
||||
}
|
||||
|
||||
// A session that still needs a code gets a short life of its own: long enough
|
||||
// to fetch a phone, not long enough to be worth stealing.
|
||||
@@ -162,6 +205,19 @@ func (s *Server) handleLoginTOTP(w http.ResponseWriter, r *http.Request) {
|
||||
user := userFrom(r.Context())
|
||||
session := sessionFrom(r.Context())
|
||||
|
||||
// Counted per account as well as per address.
|
||||
//
|
||||
// The address bucket alone is defeated by anybody with more than one, which is
|
||||
// the ordinary case — a phone and a laptop is already two. A code is six
|
||||
// digits: the only thing that makes it safe is that guesses are scarce, and
|
||||
// scarcity has to be counted against the thing being attacked.
|
||||
if ok, retry := s.totpLimiter.allow(user.ID); !ok {
|
||||
w.Header().Set("Retry-After", retryAfterSeconds(retry))
|
||||
writeError(w, http.StatusTooManyRequests, CodeRateLimited,
|
||||
"too many verification codes; wait a moment and try again")
|
||||
return
|
||||
}
|
||||
|
||||
err := auth.VerifyTOTP(user.TOTPSecret, req.Code, time.Now())
|
||||
if err != nil {
|
||||
// A recovery code is accepted in the same field. Making somebody find a
|
||||
@@ -169,6 +225,15 @@ func (s *Server) handleLoginTOTP(w http.ResponseWriter, r *http.Request) {
|
||||
// introduce a second concept.
|
||||
used, rerr := s.db.UseRecoveryCode(r.Context(), user.ID, req.Code)
|
||||
if rerr != nil || !used {
|
||||
// The pending session is left alive on purpose.
|
||||
//
|
||||
// Ending it on the first wrong digit was considered and rejected: it
|
||||
// would make somebody who mistyped one number type their password again,
|
||||
// and it buys almost nothing. What makes a six-digit code safe is that
|
||||
// guesses are scarce, and the per-account bucket above is what makes
|
||||
// them scarce — ten an hour, however many times the password is
|
||||
// re-entered. The attack this closes was never "one session, many
|
||||
// guesses"; it was "a fresh bucket for every password check".
|
||||
writeError(w, http.StatusUnauthorized, CodeUnauthorized,
|
||||
"that verification code is not valid")
|
||||
return
|
||||
@@ -176,6 +241,11 @@ func (s *Server) handleLoginTOTP(w http.ResponseWriter, r *http.Request) {
|
||||
s.log.Info("recovery code used", "user", user.ID)
|
||||
}
|
||||
|
||||
// Finished. Now the buckets may be cleared — both of them, so somebody who
|
||||
// fumbled the code twice is not still waiting once they get it right.
|
||||
s.loginLimiter.reset(clientIP(r))
|
||||
s.totpLimiter.reset(user.ID)
|
||||
|
||||
if err := s.db.PromoteSession(r.Context(), session.ID, s.cfg.SessionTTL); err != nil {
|
||||
s.internal(w, r, "promote session", err)
|
||||
return
|
||||
|
||||
@@ -27,6 +27,9 @@ type testServer struct {
|
||||
*httptest.Server
|
||||
db *store.DB
|
||||
client *http.Client
|
||||
// api er den samme server, routeren hænger på. Et par prøver har brug for at
|
||||
// nå ind i den — se `icsFetch` — og uden den her er der kun håndtaget udefra.
|
||||
api *Server
|
||||
}
|
||||
|
||||
func newTestServer(t *testing.T) *testServer {
|
||||
@@ -62,7 +65,8 @@ func newTestServerWith(t *testing.T, configure func(*config.Config)) *testServer
|
||||
// Discard: a passing test should print nothing, and these handlers log freely.
|
||||
log := slog.New(slog.NewTextHandler(io.Discard, nil))
|
||||
|
||||
srv := httptest.NewServer(New(cfg, db, log, nil))
|
||||
api := New(cfg, db, log, nil)
|
||||
srv := httptest.NewServer(api)
|
||||
t.Cleanup(srv.Close)
|
||||
|
||||
jar, err := cookiejar.New(nil)
|
||||
@@ -75,6 +79,7 @@ func newTestServerWith(t *testing.T, configure func(*config.Config)) *testServer
|
||||
// The cookie jar is what makes this a session test rather than a
|
||||
// sequence of unrelated requests.
|
||||
client: &http.Client{Jar: jar, Timeout: 10 * time.Second},
|
||||
api: api,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -301,7 +306,11 @@ func TestTOTPEnrolmentAndTwoStepLogin(t *testing.T) {
|
||||
|
||||
// Begin: a secret exists, but 2FA is not on yet — closing the tab here must
|
||||
// not lock the account.
|
||||
resp, body := ts.do(t, "POST", "/api/v1/auth/totp/setup", nil)
|
||||
// Med adgangskoden: at slå andenfaktoren til er mindst lige så indgribende som
|
||||
// at slå den fra, og den har altid spurgt.
|
||||
resp, body := ts.do(t, "POST", "/api/v1/auth/totp/setup", map[string]string{
|
||||
"password": "et langt kodeord",
|
||||
})
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
t.Fatalf("totp setup: %d %v", resp.StatusCode, body)
|
||||
}
|
||||
@@ -368,7 +377,9 @@ func TestRecoveryCodeCompletesLoginOnce(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
ts.bootstrap(t)
|
||||
|
||||
_, body := ts.do(t, "POST", "/api/v1/auth/totp/setup", nil)
|
||||
_, body := ts.do(t, "POST", "/api/v1/auth/totp/setup", map[string]string{
|
||||
"password": "et langt kodeord",
|
||||
})
|
||||
secret := body["secret"].(string)
|
||||
_, body = ts.do(t, "POST", "/api/v1/auth/totp/confirm", map[string]string{
|
||||
"code": codeFor(t, secret),
|
||||
@@ -897,3 +908,65 @@ func TestUserAgentsAreSummarisedForAPerson(t *testing.T) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Andenfaktoren kan ikke slås til uden adgangskoden.
|
||||
//
|
||||
// Døren åbnede før med et token og lukkede kun med et kodeord: `disable` har
|
||||
// altid spurgt, `setup` spurgte om ingenting. Et stjålet token kunne derfor hente
|
||||
// hemmeligheden, lægge den i sin egen godkender, bekræfte den og beholde den
|
||||
// eneste kopi af gendannelseskoderne — hvorefter ejerens næste login stoppede ved
|
||||
// en kode, de ikke havde, og de ikke kunne slå den fra uden at logge ind først.
|
||||
func TestTOTPCannotBeSwitchedOnWithoutThePassword(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
ts.bootstrap(t)
|
||||
|
||||
resp, _ := ts.do(t, "POST", "/api/v1/auth/totp/setup", map[string]string{
|
||||
"password": "det forkerte kodeord",
|
||||
})
|
||||
if resp.StatusCode != http.StatusUnprocessableEntity {
|
||||
t.Fatalf("status %d, ventede at en forkert adgangskode blev afvist", resp.StatusCode)
|
||||
}
|
||||
|
||||
resp, _ = ts.do(t, "POST", "/api/v1/auth/totp/setup", map[string]string{})
|
||||
if resp.StatusCode == http.StatusOK {
|
||||
t.Fatal("en tom adgangskode blev accepteret")
|
||||
}
|
||||
}
|
||||
|
||||
// Ratebegrænseren må ikke nulstilles af adgangskoden alene.
|
||||
//
|
||||
// Den blev det, og det gav én, der havde kodeordet, ubegrænsede portioner på ti
|
||||
// gæt: log ind, få en frisk spand og en frisk ventende session, brug ti gæt på
|
||||
// koden, log ind igen. Andenfaktoren findes præcis til det tilfælde, hvor
|
||||
// kodeordet allerede er tabt, så den portion skal betales for hver gang.
|
||||
func TestAPasswordAloneDoesNotRefillTheCodeAttempts(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
ts.bootstrap(t)
|
||||
|
||||
_, body := ts.do(t, "POST", "/api/v1/auth/totp/setup", map[string]string{
|
||||
"password": "et langt kodeord",
|
||||
})
|
||||
secret := body["secret"].(string)
|
||||
ts.do(t, "POST", "/api/v1/auth/totp/confirm", map[string]string{"code": codeFor(t, secret)})
|
||||
ts.do(t, "POST", "/api/v1/auth/logout", nil)
|
||||
|
||||
// Kodeordet er rigtigt hver gang; kun koden er forkert. Uden loftet ville det
|
||||
// her kunne blive ved i det uendelige.
|
||||
refused := 0
|
||||
for i := 0; i < 14; i++ {
|
||||
resp, _ := ts.do(t, "POST", "/api/v1/auth/login", map[string]string{
|
||||
"email": "[email protected]", "password": "et langt kodeord",
|
||||
})
|
||||
if resp.StatusCode == http.StatusTooManyRequests {
|
||||
refused++
|
||||
continue
|
||||
}
|
||||
resp, _ = ts.do(t, "POST", "/api/v1/auth/login/totp", map[string]string{"code": "000000"})
|
||||
if resp.StatusCode == http.StatusTooManyRequests {
|
||||
refused++
|
||||
}
|
||||
}
|
||||
if refused == 0 {
|
||||
t.Fatal("fjorten runder med forkerte koder blev aldrig afvist; spanden fyldes stadig af adgangskoden")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,297 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/kristianwind/verdande/internal/beacon"
|
||||
"github.com/kristianwind/verdande/internal/safedial"
|
||||
"github.com/kristianwind/verdande/internal/store"
|
||||
)
|
||||
|
||||
// The beacon, from the instance's side.
|
||||
//
|
||||
// Two halves that happen to live in the same binary: the part that reports in,
|
||||
// which every installation runs, and the part that receives, which only the one
|
||||
// acting as collector turns on. See internal/beacon for what is sent and why it
|
||||
// is on by default rather than compulsory.
|
||||
|
||||
const beaconScope = "beacon"
|
||||
|
||||
// beaconConfig is what this instance has been told to do. The zero value is what
|
||||
// a fresh installation gets, which is why Enabled defaults to true elsewhere and
|
||||
// not here — a zero struct must never mean "silently on".
|
||||
type beaconConfig struct {
|
||||
Enabled bool `json:"enabled"`
|
||||
Collector string `json:"collector_url"`
|
||||
InstanceID string `json:"instance_id"`
|
||||
IsCollector bool `json:"is_collector"`
|
||||
PublishCount bool `json:"publish_count"`
|
||||
LastPingAt int64 `json:"last_ping_at,omitempty"`
|
||||
}
|
||||
|
||||
// beaconSettings reads the configuration, filling in the defaults and minting the
|
||||
// anonymous id on first use.
|
||||
//
|
||||
// The id is generated here rather than derived from anything about the machine.
|
||||
// A hash of the hostname or the base URL would be stable and convenient and would
|
||||
// also be a value the collector could compare against a guess — which is the
|
||||
// difference between anonymous and pseudonymous, and it is the whole point.
|
||||
func (s *Server) beaconSettings(ctx context.Context) (beaconConfig, error) {
|
||||
cfg := beaconConfig{Enabled: true, Collector: beacon.DefaultCollector}
|
||||
|
||||
stored, err := s.db.InstanceSettings(ctx, beaconScope)
|
||||
if err != nil {
|
||||
return cfg, err
|
||||
}
|
||||
|
||||
if v, ok := stored["enabled"].(bool); ok {
|
||||
cfg.Enabled = v
|
||||
}
|
||||
if v, ok := stored["collector_url"].(string); ok && strings.TrimSpace(v) != "" {
|
||||
cfg.Collector = strings.TrimSpace(v)
|
||||
}
|
||||
if v, ok := stored["instance_id"].(string); ok && beacon.ValidID(v) {
|
||||
cfg.InstanceID = v
|
||||
}
|
||||
if v, ok := stored["is_collector"].(bool); ok {
|
||||
cfg.IsCollector = v
|
||||
}
|
||||
if v, ok := stored["publish_count"].(bool); ok {
|
||||
cfg.PublishCount = v
|
||||
}
|
||||
if v, ok := stored["last_ping_at"].(float64); ok {
|
||||
cfg.LastPingAt = int64(v)
|
||||
}
|
||||
|
||||
if cfg.InstanceID == "" {
|
||||
cfg.InstanceID = store.NewID()
|
||||
if err := s.saveBeaconSettings(ctx, cfg); err != nil {
|
||||
return cfg, err
|
||||
}
|
||||
}
|
||||
return cfg, nil
|
||||
}
|
||||
|
||||
func (s *Server) saveBeaconSettings(ctx context.Context, cfg beaconConfig) error {
|
||||
return s.db.SetInstanceSettings(ctx, beaconScope, map[string]any{
|
||||
"enabled": cfg.Enabled,
|
||||
"collector_url": cfg.Collector,
|
||||
"instance_id": cfg.InstanceID,
|
||||
"is_collector": cfg.IsCollector,
|
||||
"publish_count": cfg.PublishCount,
|
||||
"last_ping_at": cfg.LastPingAt,
|
||||
})
|
||||
}
|
||||
|
||||
// --- what the settings page shows ------------------------------------------------
|
||||
|
||||
type beaconStatusJSON struct {
|
||||
Enabled bool `json:"enabled"`
|
||||
Collector string `json:"collector_url"`
|
||||
InstanceID string `json:"instance_id"`
|
||||
Version string `json:"version"`
|
||||
IsCollector bool `json:"is_collector"`
|
||||
PublishCount bool `json:"publish_count"`
|
||||
LastPingAt string `json:"last_ping_at,omitempty"`
|
||||
Stats *store.BeaconStats `json:"stats,omitempty"`
|
||||
}
|
||||
|
||||
func (s *Server) handleBeaconStatus(w http.ResponseWriter, r *http.Request) {
|
||||
cfg, err := s.beaconSettings(r.Context())
|
||||
if err != nil {
|
||||
s.internal(w, r, "beacon settings", err)
|
||||
return
|
||||
}
|
||||
|
||||
out := beaconStatusJSON{
|
||||
Enabled: cfg.Enabled, Collector: cfg.Collector, InstanceID: cfg.InstanceID,
|
||||
Version: Version, IsCollector: cfg.IsCollector, PublishCount: cfg.PublishCount,
|
||||
}
|
||||
if cfg.LastPingAt > 0 {
|
||||
out.LastPingAt = time.Unix(cfg.LastPingAt, 0).Format(time.RFC3339)
|
||||
}
|
||||
if cfg.IsCollector {
|
||||
stats, err := s.db.BeaconSummary(r.Context())
|
||||
if err != nil {
|
||||
s.internal(w, r, "beacon summary", err)
|
||||
return
|
||||
}
|
||||
out.Stats = &stats
|
||||
}
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
}
|
||||
|
||||
type beaconRequest struct {
|
||||
Enabled *bool `json:"enabled"`
|
||||
Collector *string `json:"collector_url"`
|
||||
IsCollector *bool `json:"is_collector"`
|
||||
PublishCount *bool `json:"publish_count"`
|
||||
}
|
||||
|
||||
func (s *Server) handleSetBeacon(w http.ResponseWriter, r *http.Request) {
|
||||
var req beaconRequest
|
||||
if err := decodeJSON(w, r, &req); err != nil {
|
||||
return
|
||||
}
|
||||
cfg, err := s.beaconSettings(r.Context())
|
||||
if err != nil {
|
||||
s.internal(w, r, "beacon settings", err)
|
||||
return
|
||||
}
|
||||
|
||||
if req.Enabled != nil {
|
||||
cfg.Enabled = *req.Enabled
|
||||
}
|
||||
if req.IsCollector != nil {
|
||||
cfg.IsCollector = *req.IsCollector
|
||||
}
|
||||
if req.PublishCount != nil {
|
||||
cfg.PublishCount = *req.PublishCount
|
||||
}
|
||||
if req.Collector != nil {
|
||||
url := strings.TrimSpace(*req.Collector)
|
||||
if url == "" {
|
||||
url = beacon.DefaultCollector
|
||||
}
|
||||
// http and https only, and said as a field error rather than a 500: an
|
||||
// operator pointing this at their own collector will mistype it, and the
|
||||
// page should say which field is wrong.
|
||||
if !strings.HasPrefix(url, "http://") && !strings.HasPrefix(url, "https://") {
|
||||
writeFieldErrors(w, map[string]string{"collector_url": "must start with http:// or https://"})
|
||||
return
|
||||
}
|
||||
cfg.Collector = url
|
||||
}
|
||||
|
||||
if err := s.saveBeaconSettings(r.Context(), cfg); err != nil {
|
||||
s.internal(w, r, "save beacon settings", err)
|
||||
return
|
||||
}
|
||||
s.handleBeaconStatus(w, r)
|
||||
}
|
||||
|
||||
// --- the collector's side ---------------------------------------------------------
|
||||
|
||||
// handleBeaconPing receives one report. Unauthenticated, because the whole point
|
||||
// is that a stranger's installation can reach it.
|
||||
//
|
||||
// Nothing about the request is recorded except the two values in the body. The
|
||||
// source address is not read here, and there is no code path from this function
|
||||
// to anything that could write one down — which is the only version of that
|
||||
// promise worth making.
|
||||
func (s *Server) handleBeaconPing(w http.ResponseWriter, r *http.Request) {
|
||||
cfg, err := s.beaconSettings(r.Context())
|
||||
if err != nil {
|
||||
s.internal(w, r, "beacon settings", err)
|
||||
return
|
||||
}
|
||||
// An instance that is not a collector does not quietly accept and drop pings —
|
||||
// it says there is nothing here. Otherwise every installation in the world is a
|
||||
// spam sink that answers 200 to anything.
|
||||
if !cfg.IsCollector {
|
||||
writeError(w, http.StatusNotFound, CodeNotFound, "this instance is not a beacon collector")
|
||||
return
|
||||
}
|
||||
|
||||
r.Body = http.MaxBytesReader(w, r.Body, 4<<10)
|
||||
var p beacon.Payload
|
||||
if err := decodeJSON(w, r, &p); err != nil {
|
||||
return
|
||||
}
|
||||
|
||||
// Checked, not trusted. This is the one endpoint anybody can post to, and the
|
||||
// two columns behind it should only ever hold the two shapes this program
|
||||
// writes — an id that is not id-shaped is somebody probing, not an install.
|
||||
if !beacon.ValidID(p.InstanceID) {
|
||||
writeFieldErrors(w, map[string]string{"instance_id": "malformed"})
|
||||
return
|
||||
}
|
||||
if p.Version != "" && !beacon.ValidVersion(p.Version) {
|
||||
writeFieldErrors(w, map[string]string{"version": "malformed"})
|
||||
return
|
||||
}
|
||||
|
||||
if err := s.db.RecordBeacon(r.Context(), p.InstanceID, p.Version); err != nil {
|
||||
s.internal(w, r, "record beacon", err)
|
||||
return
|
||||
}
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
}
|
||||
|
||||
// handleBeaconCount publishes the number, when the collector has been told to.
|
||||
//
|
||||
// Thirty days rather than a running total: a total only climbs, so it stops
|
||||
// meaning "how many people use this" and starts meaning "how many ever tried it".
|
||||
func (s *Server) handleBeaconCount(w http.ResponseWriter, r *http.Request) {
|
||||
cfg, err := s.beaconSettings(r.Context())
|
||||
if err != nil {
|
||||
s.internal(w, r, "beacon settings", err)
|
||||
return
|
||||
}
|
||||
if !cfg.IsCollector || !cfg.PublishCount {
|
||||
writeError(w, http.StatusNotFound, CodeNotFound, "not published")
|
||||
return
|
||||
}
|
||||
|
||||
n, err := s.db.BeaconCount(r.Context(), 30*24*time.Hour)
|
||||
if err != nil {
|
||||
s.internal(w, r, "beacon count", err)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Cache-Control", "public, max-age=60")
|
||||
writeJSON(w, http.StatusOK, map[string]any{"installs": n, "window_days": 30})
|
||||
}
|
||||
|
||||
// SendBeacon reports this installation, at most once a day.
|
||||
//
|
||||
// Exported and handed to the jobs runner the same way the mailbox syncs are: the
|
||||
// settings, the anonymous id and the version all live on this side, and the
|
||||
// runner should not have to know about any of them.
|
||||
//
|
||||
// The day is counted from the last successful send, not from a wall-clock hour.
|
||||
// A fleet of installations that all pinged at 03:00 would arrive in one spike;
|
||||
// counted this way they spread themselves out by when each was started.
|
||||
func (s *Server) SendBeacon(ctx context.Context) error {
|
||||
cfg, err := s.beaconSettings(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !cfg.Enabled {
|
||||
return nil
|
||||
}
|
||||
// The collector does not report to itself over the network. It would work, and
|
||||
// it would also mean the one instance whose count matters most depends on its
|
||||
// own tunnel being up to be counted.
|
||||
if cfg.IsCollector {
|
||||
if err := s.db.RecordBeacon(ctx, cfg.InstanceID, Version); err != nil {
|
||||
return err
|
||||
}
|
||||
cfg.LastPingAt = time.Now().Unix()
|
||||
return s.saveBeaconSettings(ctx, cfg)
|
||||
}
|
||||
|
||||
if time.Since(time.Unix(cfg.LastPingAt, 0)) < 24*time.Hour {
|
||||
return nil
|
||||
}
|
||||
|
||||
// safedial, ikke http.DefaultClient. Adressen kan skrives i fladen, og en
|
||||
// administrator, der peger den på 127.0.0.1 eller på 169.254.169.254, ville
|
||||
// ellers have en maskine, der henter indefra på kommando — og svaret på om den
|
||||
// svarede, er i sig selv det, sådan et forsøg leder efter.
|
||||
if err := beacon.Send(ctx, safedial.Client(15*time.Second), cfg.Collector, beacon.Payload{
|
||||
InstanceID: cfg.InstanceID,
|
||||
Version: Version,
|
||||
}); err != nil {
|
||||
// Logged at info, not error. The collector being unreachable is not this
|
||||
// installation's problem and must never look like one — it is somebody
|
||||
// else's server, and the only consequence is a number being one lower.
|
||||
s.log.Info("beacon not delivered", "err", err)
|
||||
return nil
|
||||
}
|
||||
|
||||
cfg.LastPingAt = time.Now().Unix()
|
||||
return s.saveBeaconSettings(ctx, cfg)
|
||||
}
|
||||
@@ -231,6 +231,33 @@ func (s *Server) handleCalDAVPut(w http.ResponseWriter, r *http.Request) {
|
||||
|
||||
existing, getErr := s.db.GetTask(r.Context(), taskID, user.ID)
|
||||
if getErr == nil {
|
||||
// The task decides, not the URL.
|
||||
//
|
||||
// The check above is about the project the *client named*, and a client
|
||||
// names both halves of this address. Asking the project for permission and
|
||||
// then taking the task id from the same string ties nothing together: a
|
||||
// viewer on a shared project could name a project of their own — the Inbox,
|
||||
// which everybody owns and may edit — and put the shared task's id in the
|
||||
// filename. The project check passed on the Inbox and the write landed on a
|
||||
// task they may only read.
|
||||
//
|
||||
// GetTask does not close it either: it answers "may this person see it",
|
||||
// and a viewer may. `CanEdit` is the question, and it is the same one
|
||||
// handleUpdateTask asks — the API door has always asked it.
|
||||
role, err := store.TaskRole(r.Context(), s.db, taskID, user.ID)
|
||||
if err != nil || !role.CanEdit() {
|
||||
http.Error(w, "forbidden", http.StatusForbidden)
|
||||
return
|
||||
}
|
||||
// And it has to live where the address says it does. Without this an editor
|
||||
// on two projects can reach either through the other's URL — nothing is
|
||||
// escalated by that on its own, but a URL allowed to lie about where a
|
||||
// resource lives is how the case above got in.
|
||||
if existing.ProjectID != projectID {
|
||||
http.NotFound(w, r)
|
||||
return
|
||||
}
|
||||
|
||||
// An update. Only the fields a task client can express are touched;
|
||||
// everything else the task carries stays as it is.
|
||||
content := parsed.Summary
|
||||
|
||||
@@ -0,0 +1,808 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/go-chi/chi/v5"
|
||||
|
||||
"github.com/kristianwind/verdande/internal/gcal"
|
||||
"github.com/kristianwind/verdande/internal/ics"
|
||||
"github.com/kristianwind/verdande/internal/safedial"
|
||||
"github.com/kristianwind/verdande/internal/store"
|
||||
)
|
||||
|
||||
// defaultCalendarSyncBudget bounds one "fetch now" when nothing else says
|
||||
// otherwise. Well under the hundred seconds a proxy is willing to wait, so this
|
||||
// server is the one that answers a slow Google rather than Cloudflare's own HTML
|
||||
// page — which is how a failure arrives with no code, no message and nothing in
|
||||
// the error log.
|
||||
const defaultCalendarSyncBudget = 25 * time.Second
|
||||
|
||||
// The window verdande keeps a copy of.
|
||||
//
|
||||
// A month grid pages, so asking Google on every arrow press would mean a round trip
|
||||
// for a question answered ninety seconds ago — a calendar slower than the calendar
|
||||
// it is showing. A cached window is the alternative, and a window has to end
|
||||
// somewhere. Ninety days back is roughly "when was that", a year forward is past
|
||||
// anything anybody has booked; between them they cover every screen the grid can
|
||||
// reach without a person noticing an edge.
|
||||
//
|
||||
// The edge is still real, and the events endpoint says where it is rather than
|
||||
// answering an empty list for a month outside it. A calendar that quietly shows no
|
||||
// events is a calendar that lies about a day being clear.
|
||||
const (
|
||||
calendarWindowBack = 90
|
||||
calendarWindowAhead = 365
|
||||
calendarDateLayout = "2006-01-02"
|
||||
calendarProviderName = "google"
|
||||
)
|
||||
|
||||
func (s *Server) calendarClient(accessToken string) *gcal.Client {
|
||||
return gcal.NewClient(accessToken).At(s.cfg.CalendarAPIURL)
|
||||
}
|
||||
|
||||
// calendarConfig is the same registration Gmail signs in through, asking for a
|
||||
// different scope and coming back to a different URI.
|
||||
//
|
||||
// One OAuth client, two features. Google issues a refresh token per authorisation
|
||||
// and keeps up to a hundred of them live per account, each carrying the scopes it
|
||||
// was granted with — so connecting the calendar does not disturb a working Gmail,
|
||||
// and disconnecting one does not take the other with it.
|
||||
func (s *Server) calendarConfig(ctx context.Context) gcal.Config {
|
||||
cfg := s.gmailConfig(ctx)
|
||||
cfg.Scope = gcal.Scope
|
||||
cfg.RedirectURL = s.cfg.CalendarRedirectURL()
|
||||
return cfg
|
||||
}
|
||||
|
||||
// --- connecting ------------------------------------------------------------------
|
||||
|
||||
// handleCalendarAuthorize starts the flow. Same shape as Gmail's, and the same
|
||||
// reason for keeping the verifier server-side: the callback arrives as a top-level
|
||||
// navigation from Google, and a SameSite=Lax cookie is the kind of thing that works
|
||||
// until a browser tightens its defaults.
|
||||
func (s *Server) handleCalendarAuthorize(w http.ResponseWriter, r *http.Request) {
|
||||
user := userFrom(r.Context())
|
||||
cfg := s.calendarConfig(r.Context())
|
||||
|
||||
if !cfg.Configured() {
|
||||
writeError(w, http.StatusConflict, CodeGmailNotConfigured,
|
||||
"no Google OAuth client is registered on this server")
|
||||
return
|
||||
}
|
||||
|
||||
pkce, err := gcal.NewPKCE()
|
||||
if err != nil {
|
||||
s.internal(w, r, "calendar pkce", err)
|
||||
return
|
||||
}
|
||||
|
||||
// Its own settings scope, not Gmail's. Two handshakes in one row would mean a
|
||||
// person who started both flows had one verifier, and whichever came back
|
||||
// second would be refused for a reason nothing on the screen could explain.
|
||||
settings, err := s.db.UserSettings(r.Context(), user.ID, "calendar")
|
||||
if err != nil {
|
||||
s.internal(w, r, "calendar settings", err)
|
||||
return
|
||||
}
|
||||
settings["pkce_verifier"] = pkce.Verifier
|
||||
settings["pkce_state"] = pkce.State
|
||||
// The attempt expires: a stored verifier that lives forever is a replay window
|
||||
// that lives forever.
|
||||
settings["pkce_expires"] = time.Now().Add(10 * time.Minute).Unix()
|
||||
|
||||
if err := s.db.SetUserSettings(r.Context(), user.ID, "calendar", settings); err != nil {
|
||||
s.internal(w, r, "calendar settings", err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]string{"url": cfg.AuthURL(pkce)})
|
||||
}
|
||||
|
||||
// handleCalendarCallback is where Google sends the browser back.
|
||||
//
|
||||
// It answers with a redirect into the app rather than with JSON: the person is
|
||||
// looking at a browser tab, not at a fetch response.
|
||||
func (s *Server) handleCalendarCallback(w http.ResponseWriter, r *http.Request) {
|
||||
user := userFrom(r.Context())
|
||||
settingsURL := s.cfg.BaseURL + "/indstillinger/integrationer"
|
||||
|
||||
if errParam := r.URL.Query().Get("error"); errParam != "" {
|
||||
// Somebody clicked "cancel" on the consent screen — or, on an Internal
|
||||
// OAuth app, Google refused an account outside the organisation with
|
||||
// `org_internal`. Both arrive here, and the page names what it was given.
|
||||
http.Redirect(w, r, settingsURL+"?calendar="+errParam, http.StatusFound)
|
||||
return
|
||||
}
|
||||
|
||||
code := r.URL.Query().Get("code")
|
||||
state := r.URL.Query().Get("state")
|
||||
if code == "" || state == "" {
|
||||
http.Redirect(w, r, settingsURL+"?calendar=invalid", http.StatusFound)
|
||||
return
|
||||
}
|
||||
|
||||
settings, err := s.db.UserSettings(r.Context(), user.ID, "calendar")
|
||||
if err != nil {
|
||||
s.internal(w, r, "calendar settings", err)
|
||||
return
|
||||
}
|
||||
verifier, _ := settings["pkce_verifier"].(string)
|
||||
expected, _ := settings["pkce_state"].(string)
|
||||
expires, _ := settings["pkce_expires"].(float64)
|
||||
|
||||
// The state has to match, or this endpoint accepts a code from anywhere.
|
||||
if verifier == "" || expected == "" || state != expected {
|
||||
http.Redirect(w, r, settingsURL+"?calendar=state", http.StatusFound)
|
||||
return
|
||||
}
|
||||
if int64(expires) < time.Now().Unix() {
|
||||
http.Redirect(w, r, settingsURL+"?calendar=expired", http.StatusFound)
|
||||
return
|
||||
}
|
||||
|
||||
token, err := s.calendarConfig(r.Context()).Exchange(r.Context(), code,
|
||||
gcal.PKCE{Verifier: verifier})
|
||||
if err != nil {
|
||||
s.log.Error("calendar exchange", "err", err, "user", user.ID)
|
||||
http.Redirect(w, r, settingsURL+"?calendar=failed", http.StatusFound)
|
||||
return
|
||||
}
|
||||
if token.RefreshToken == "" {
|
||||
// Without one the connection dies in an hour. Google only withholds it when
|
||||
// the consent prompt was skipped, which access_type and prompt are meant to
|
||||
// prevent — so this means something is wrong with the client registration
|
||||
// rather than with this request.
|
||||
s.log.Error("calendar returned no refresh token", "user", user.ID)
|
||||
http.Redirect(w, r, settingsURL+"?calendar=norefresh", http.StatusFound)
|
||||
return
|
||||
}
|
||||
|
||||
// The one-time values are cleared: keeping a spent verifier serves nothing and
|
||||
// is one more secret at rest.
|
||||
delete(settings, "pkce_verifier")
|
||||
delete(settings, "pkce_state")
|
||||
delete(settings, "pkce_expires")
|
||||
if err := s.db.SetUserSettings(r.Context(), user.ID, "calendar", settings); err != nil {
|
||||
s.internal(w, r, "clear calendar handshake", err)
|
||||
return
|
||||
}
|
||||
|
||||
account := &store.CalendarAccount{
|
||||
UserID: user.ID, Provider: calendarProviderName,
|
||||
RefreshToken: token.RefreshToken, AccessToken: token.AccessToken,
|
||||
ExpiresAt: token.ExpiresAt,
|
||||
}
|
||||
if err := s.db.SaveCalendarAccount(r.Context(), account); err != nil {
|
||||
s.internal(w, r, "save calendar tokens", err)
|
||||
return
|
||||
}
|
||||
|
||||
// The list of calendars, fetched now so the page that opens next has something
|
||||
// to tick rather than an empty panel and a button. Best effort: an account that
|
||||
// connects but cannot list itself is still connected, and the status endpoint
|
||||
// asks again when it finds nothing.
|
||||
if _, err := s.refreshCalendarList(r.Context(), account); err != nil {
|
||||
s.log.Warn("calendar list", "err", err, "user", user.ID)
|
||||
}
|
||||
|
||||
s.log.Info("calendar connected", "user", user.ID, "account", account.Account)
|
||||
http.Redirect(w, r, settingsURL+"?calendar=connected", http.StatusFound)
|
||||
}
|
||||
|
||||
// --- what the settings page reads --------------------------------------------------
|
||||
|
||||
type calendarStatusJSON struct {
|
||||
Connected bool `json:"connected"`
|
||||
Account string `json:"account,omitempty"`
|
||||
Calendars []store.Calendar `json:"calendars"`
|
||||
// RedirectURI is what has to be registered with Google, spelled out. Derived
|
||||
// from VERDANDE_BASE_URL and matched exactly — the single most likely thing to
|
||||
// be wrong, and the error Google gives for it names neither value.
|
||||
RedirectURI string `json:"redirect_uri"`
|
||||
// HasClient says whether there is an OAuth registration to connect through at
|
||||
// all, so the page can explain rather than offer a button that answers 409.
|
||||
HasClient bool `json:"has_client"`
|
||||
LastSyncAt string `json:"last_sync_at,omitempty"`
|
||||
// ReadOnly is stated rather than assumed. Nothing writes a Google calendar, and
|
||||
// an interface that quietly looks editable is one somebody will try to edit.
|
||||
ReadOnly bool `json:"read_only"`
|
||||
|
||||
// Abonnementer står ved siden af Google-kontoen og ikke inde i den: man kan
|
||||
// have det ene uden det andet, og det er hele pointen med dem — en privat
|
||||
// konto, der ikke kan bruge en Internal OAuth-klient, kan abonnere.
|
||||
Subscriptions []store.CalendarAccount `json:"subscriptions"`
|
||||
}
|
||||
|
||||
func (s *Server) handleGetCalendar(w http.ResponseWriter, r *http.Request) {
|
||||
// Læst først, så de også kommer med, når der ingen Google-konto er.
|
||||
subscriptions, err := s.db.Subscriptions(r.Context(), userFrom(r.Context()).ID)
|
||||
if err != nil {
|
||||
s.internal(w, r, "subscriptions", err)
|
||||
return
|
||||
}
|
||||
|
||||
user := userFrom(r.Context())
|
||||
|
||||
out := calendarStatusJSON{
|
||||
Calendars: []store.Calendar{},
|
||||
RedirectURI: s.cfg.CalendarRedirectURL(),
|
||||
HasClient: s.calendarConfig(r.Context()).Configured(),
|
||||
ReadOnly: true,
|
||||
}
|
||||
|
||||
account, err := s.db.CalendarAccountFor(r.Context(), user.ID, calendarProviderName)
|
||||
if err != nil {
|
||||
s.internal(w, r, "calendar account", err)
|
||||
return
|
||||
}
|
||||
if account == nil || account.RefreshToken == "" {
|
||||
out.Subscriptions = subscriptions
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
return
|
||||
}
|
||||
|
||||
out.Connected = true
|
||||
out.Account = account.Account
|
||||
if !account.LastSyncAt.IsZero() {
|
||||
out.LastSyncAt = account.LastSyncAt.Format(time.RFC3339)
|
||||
}
|
||||
|
||||
calendars, err := s.db.Calendars(r.Context(), account.ID)
|
||||
if err != nil {
|
||||
s.internal(w, r, "calendars", err)
|
||||
return
|
||||
}
|
||||
// An account connected during a Google hiccup has no calendars and no way back:
|
||||
// the person sees an empty list and nothing to press. Asking again here costs
|
||||
// one request on a page nobody opens often, and means the wrong answer lasts
|
||||
// until the next look rather than for ever. Same reasoning as the Gmail address.
|
||||
if len(calendars) == 0 {
|
||||
if fetched, err := s.refreshCalendarList(r.Context(), account); err == nil {
|
||||
calendars = fetched
|
||||
out.Account = account.Account
|
||||
} else {
|
||||
s.log.Warn("calendar list", "err", err, "user", user.ID)
|
||||
}
|
||||
}
|
||||
if calendars != nil {
|
||||
out.Calendars = calendars
|
||||
}
|
||||
out.Subscriptions = subscriptions
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
}
|
||||
|
||||
type showCalendarsRequest struct {
|
||||
// Shown is the whole set, not one flip. Same reasoning as the projects' order:
|
||||
// a person has a handful of calendars, the question is "these ones", and a list
|
||||
// that can only be built by several requests can land half applied.
|
||||
Shown []string `json:"shown"`
|
||||
}
|
||||
|
||||
func (s *Server) handleSetCalendars(w http.ResponseWriter, r *http.Request) {
|
||||
var req showCalendarsRequest
|
||||
if err := decodeJSON(w, r, &req); err != nil {
|
||||
return
|
||||
}
|
||||
user := userFrom(r.Context())
|
||||
|
||||
account, err := s.db.CalendarAccountFor(r.Context(), user.ID, calendarProviderName)
|
||||
if err != nil {
|
||||
s.internal(w, r, "calendar account", err)
|
||||
return
|
||||
}
|
||||
if account == nil {
|
||||
// A 404 rather than a 409: everything the caller may not see answers 404,
|
||||
// and "there is no such connection" is the true answer either way.
|
||||
writeError(w, http.StatusNotFound, CodeNotFound, "no calendar is connected")
|
||||
return
|
||||
}
|
||||
if err := s.db.ShowCalendars(r.Context(), account.ID, req.Shown); err != nil {
|
||||
s.internal(w, r, "show calendars", err)
|
||||
return
|
||||
}
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
}
|
||||
|
||||
// handleDisconnectCalendar forgets the connection, and the copy of the calendars
|
||||
// with it. Unlike the tasks a mailbox made, none of this is the person's own work.
|
||||
func (s *Server) handleDisconnectCalendar(w http.ResponseWriter, r *http.Request) {
|
||||
user := userFrom(r.Context())
|
||||
account, err := s.db.CalendarAccountFor(r.Context(), user.ID, calendarProviderName)
|
||||
if err != nil {
|
||||
s.internal(w, r, "calendar account", err)
|
||||
return
|
||||
}
|
||||
if account != nil {
|
||||
if err := s.db.DeleteCalendarAccount(r.Context(), user.ID, account.ID); err != nil {
|
||||
s.internal(w, r, "disconnect calendar", err)
|
||||
return
|
||||
}
|
||||
}
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
}
|
||||
|
||||
// --- what the grid reads -----------------------------------------------------------
|
||||
|
||||
type calendarEventsJSON struct {
|
||||
Events []store.CalendarEvent `json:"events"`
|
||||
// From and To are the window verdande holds a copy of, not the window that was
|
||||
// asked for. A grid that has paged past it gets no events, and an empty answer
|
||||
// with no explanation is a calendar quietly claiming a day is clear.
|
||||
From string `json:"from"`
|
||||
To string `json:"to"`
|
||||
}
|
||||
|
||||
func (s *Server) handleCalendarEvents(w http.ResponseWriter, r *http.Request) {
|
||||
user := userFrom(r.Context())
|
||||
from := r.URL.Query().Get("from")
|
||||
to := r.URL.Query().Get("to")
|
||||
|
||||
fields := map[string]string{}
|
||||
if !validDay(from) {
|
||||
fields["from"] = "must be a date, as 2026-08-20"
|
||||
}
|
||||
if !validDay(to) {
|
||||
fields["to"] = "must be a date, as 2026-08-20"
|
||||
}
|
||||
if len(fields) > 0 {
|
||||
writeFieldErrors(w, fields)
|
||||
return
|
||||
}
|
||||
|
||||
events, err := s.db.CalendarEvents(r.Context(), user.ID, from, to)
|
||||
if err != nil {
|
||||
s.internal(w, r, "calendar events", err)
|
||||
return
|
||||
}
|
||||
if events == nil {
|
||||
events = []store.CalendarEvent{}
|
||||
}
|
||||
windowFrom, windowTo := calendarWindow(time.Now())
|
||||
writeJSON(w, http.StatusOK, calendarEventsJSON{
|
||||
Events: events, From: windowFrom, To: windowTo,
|
||||
})
|
||||
}
|
||||
|
||||
// handleCalendarSyncNow refreshes immediately, so somebody who has just connected
|
||||
// can see it work rather than wondering whether it did.
|
||||
func (s *Server) handleCalendarSyncNow(w http.ResponseWriter, r *http.Request) {
|
||||
user := userFrom(r.Context())
|
||||
|
||||
// A budget of its own, well under what sits in front of this server. An unset
|
||||
// budget is not a budget of nothing: a zero here would expire the context
|
||||
// before the first call and make every sync a silent no-op.
|
||||
budget := s.cfg.CalendarSyncBudget
|
||||
if budget <= 0 {
|
||||
budget = defaultCalendarSyncBudget
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(r.Context(), budget)
|
||||
defer cancel()
|
||||
|
||||
count, err := s.SyncCalendars(ctx, user)
|
||||
if err != nil {
|
||||
if errors.Is(err, context.DeadlineExceeded) {
|
||||
// Not a failure: Google is slow or busy, some calendars were read, and
|
||||
// the rest arrive on the next run. Every calendar is committed as it is
|
||||
// read, so a short budget is a shorter run rather than a lost one.
|
||||
writeJSON(w, http.StatusOK, map[string]any{"events": count, "partial": true})
|
||||
return
|
||||
}
|
||||
s.upstream(w, r, CodeCalendarFailed, "sync calendar", err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]int{"events": count})
|
||||
}
|
||||
|
||||
// SyncCalendars refreshes every calendar this person has chosen to see.
|
||||
//
|
||||
// Returns how many events were written. Also used by the background job, which is
|
||||
// why it takes a context and a user rather than a request.
|
||||
func (s *Server) SyncCalendars(ctx context.Context, user *store.User) (int, error) {
|
||||
// Abonnementer først, og deres fejl tæller ikke mod Google.
|
||||
//
|
||||
// De to har intet med hinanden at gøre ud over at ende samme sted: et
|
||||
// abonnement er en adresse, der hentes, hvor Google er et token, der udløber.
|
||||
// En kalender, hvis vært er nede, må ikke få det til at se ud som om
|
||||
// Google-forbindelsen er væk — og omvendt.
|
||||
written, err := s.syncSubscriptions(ctx, user)
|
||||
if err != nil {
|
||||
s.log.Warn("calendar subscriptions", "err", err, "user", user.ID)
|
||||
}
|
||||
|
||||
account, err := s.db.CalendarAccountFor(ctx, user.ID, calendarProviderName)
|
||||
if err != nil {
|
||||
return written, err
|
||||
}
|
||||
if account == nil || account.RefreshToken == "" {
|
||||
return written, nil // not connected; nothing to do and nothing to complain about
|
||||
}
|
||||
|
||||
// The same guard the mailboxes have, for the same reason: a press of "fetch
|
||||
// now" during the sweep would otherwise have two runs replacing the same window
|
||||
// at once, and the loser's delete would land after the winner's insert.
|
||||
unlock := s.lockSync("calendar:" + account.ID)
|
||||
defer unlock()
|
||||
if fresh, err := s.db.CalendarAccountFor(ctx, user.ID, calendarProviderName); err == nil && fresh != nil {
|
||||
account = fresh
|
||||
}
|
||||
|
||||
access, err := s.calendarAccessToken(ctx, account)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
client := s.calendarClient(access)
|
||||
|
||||
// The list first, so a calendar added or renamed since the last run is drawn
|
||||
// under its new name and one that has gone stops being drawn at all.
|
||||
calendars, err := s.calendarList(ctx, client, account)
|
||||
if err != nil {
|
||||
return 0, s.forgetIfRevoked(ctx, user, account, err)
|
||||
}
|
||||
|
||||
from, to := calendarWindow(time.Now())
|
||||
for _, calendar := range calendars {
|
||||
if !calendar.Shown {
|
||||
continue
|
||||
}
|
||||
// A spent budget ends the run rather than turning into a failed fetch per
|
||||
// calendar, logged one at a time.
|
||||
if ctx.Err() != nil {
|
||||
return written, ctx.Err()
|
||||
}
|
||||
events, err := client.Events(ctx, calendar.RemoteID, from, to)
|
||||
if err != nil {
|
||||
if errors.Is(err, gcal.ErrUnauthorized) {
|
||||
return written, s.forgetIfRevoked(ctx, user, account, err)
|
||||
}
|
||||
// One calendar somebody was unshared from must not stop the others.
|
||||
s.log.Warn("calendar events", "err", err, "calendar", calendar.Name)
|
||||
continue
|
||||
}
|
||||
|
||||
rows := make([]store.CalendarEvent, 0, len(events))
|
||||
for _, e := range events {
|
||||
rows = append(rows, store.CalendarEvent{
|
||||
RemoteID: e.ID, Summary: e.Summary,
|
||||
StartsAt: e.StartsAt, EndsAt: e.EndsAt,
|
||||
StartDay: e.StartDay, EndDay: e.EndDay,
|
||||
AllDay: e.AllDay, Location: e.Location, URL: e.URL,
|
||||
})
|
||||
}
|
||||
if err := s.db.ReplaceCalendarEvents(ctx, calendar.ID, from, to, rows); err != nil {
|
||||
return written, err
|
||||
}
|
||||
written += len(rows)
|
||||
}
|
||||
|
||||
if err := s.db.MarkCalendarSynced(ctx, account.ID, time.Now()); err != nil {
|
||||
return written, err
|
||||
}
|
||||
return written, nil
|
||||
}
|
||||
|
||||
// calendarAccessToken returns a usable access token, refreshing only when the one
|
||||
// held has actually expired.
|
||||
func (s *Server) calendarAccessToken(ctx context.Context, account *store.CalendarAccount) (string, error) {
|
||||
if account.AccessToken != "" && account.ExpiresAt.After(time.Now()) {
|
||||
return account.AccessToken, nil
|
||||
}
|
||||
token, err := s.calendarConfig(ctx).Refresh(ctx, account.RefreshToken)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
account.AccessToken = token.AccessToken
|
||||
account.ExpiresAt = token.ExpiresAt
|
||||
if err := s.db.SaveCalendarAccount(ctx, account); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return account.AccessToken, nil
|
||||
}
|
||||
|
||||
// refreshCalendarList fetches the calendar list with a token it obtains itself.
|
||||
// Used by the paths that have an account but no client yet — connecting, and the
|
||||
// status endpoint correcting an empty list.
|
||||
func (s *Server) refreshCalendarList(ctx context.Context, account *store.CalendarAccount) ([]store.Calendar, error) {
|
||||
access, err := s.calendarAccessToken(ctx, account)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return s.calendarList(ctx, s.calendarClient(access), account)
|
||||
}
|
||||
|
||||
// calendarList writes what the provider says and returns what is stored, which is
|
||||
// not the same list: the provider says what exists, and only verdande knows which
|
||||
// of them somebody chose to look at.
|
||||
func (s *Server) calendarList(ctx context.Context, client *gcal.Client, account *store.CalendarAccount) ([]store.Calendar, error) {
|
||||
remote, err := client.Calendars(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
list := make([]store.Calendar, 0, len(remote))
|
||||
for _, c := range remote {
|
||||
list = append(list, store.Calendar{
|
||||
RemoteID: c.ID, Name: c.Name, Colour: c.Colour,
|
||||
TimeZone: c.TimeZone, Primary: c.Primary, Writable: c.Owner,
|
||||
})
|
||||
// The primary calendar is the account, and its id is the address. It is the
|
||||
// only place the connected account names itself without a second scope —
|
||||
// asking for a profile scope to learn an address already in the answer would
|
||||
// be one more thing on the consent screen for nothing.
|
||||
if c.Primary && account.Account == "" {
|
||||
account.Account = c.ID
|
||||
if err := s.db.SaveCalendarAccount(ctx, account); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
}
|
||||
if err := s.db.ReplaceCalendars(ctx, account.ID, list); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return s.db.Calendars(ctx, account.ID)
|
||||
}
|
||||
|
||||
// forgetIfRevoked disconnects when Google says the grant is gone, and passes every
|
||||
// other failure through untouched.
|
||||
//
|
||||
// The distinction is the point. A revoked account polled every ten minutes for ever
|
||||
// is a log line a week long and a connection the person believes they removed; a
|
||||
// Google that is merely down is one that will answer next time.
|
||||
func (s *Server) forgetIfRevoked(ctx context.Context, user *store.User, account *store.CalendarAccount, err error) error {
|
||||
if errors.Is(err, gcal.ErrUnauthorized) {
|
||||
s.log.Info("calendar access was revoked; disconnecting", "user", user.ID)
|
||||
_ = s.db.DeleteCalendarAccount(ctx, user.ID, account.ID)
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
// calendarWindow is the span verdande keeps a copy of, measured from today.
|
||||
//
|
||||
// Built from the calendar date rather than by truncating the instant: Truncate
|
||||
// rounds against the Unix epoch, which is midnight UTC, and in Copenhagen that is
|
||||
// two in the morning — so for two hours of every summer day the window would start
|
||||
// and end a day out.
|
||||
func calendarWindow(now time.Time) (string, string) {
|
||||
day := time.Date(now.Year(), now.Month(), now.Day(), 0, 0, 0, 0, now.Location())
|
||||
return day.AddDate(0, 0, -calendarWindowBack).Format(calendarDateLayout),
|
||||
day.AddDate(0, 0, calendarWindowAhead).Format(calendarDateLayout)
|
||||
}
|
||||
|
||||
func validDay(s string) bool {
|
||||
_, err := time.Parse(calendarDateLayout, s)
|
||||
return err == nil
|
||||
}
|
||||
|
||||
// --- abonnementer -----------------------------------------------------------------
|
||||
|
||||
// maxICSBytes bounds one fetch. A published calendar of a decade of holidays is a
|
||||
// few hundred kilobytes; ten megabytes is not a calendar, and reading it into
|
||||
// memory on a schedule with nobody watching is how a homelab instance falls over
|
||||
// at three in the morning.
|
||||
const maxICSBytes = 10 << 20
|
||||
|
||||
// normaliseCalendarURL turns what somebody pastes into something that can be
|
||||
// fetched.
|
||||
//
|
||||
// `webcal://` is not a scheme anything speaks. It is a convention — the same URL
|
||||
// over https, with a scheme that makes a desktop hand it to the calendar app
|
||||
// instead of the browser — and every publisher writes their links that way. A
|
||||
// person pasting one has pasted the right address; refusing it would be refusing
|
||||
// them on a technicality.
|
||||
func normaliseCalendarURL(raw string) (string, error) {
|
||||
raw = strings.TrimSpace(raw)
|
||||
if after, ok := strings.CutPrefix(raw, "webcal://"); ok {
|
||||
raw = "https://" + after
|
||||
}
|
||||
if !strings.HasPrefix(raw, "https://") && !strings.HasPrefix(raw, "http://") {
|
||||
return "", errors.New("must be an https:// or webcal:// address")
|
||||
}
|
||||
// The same wall the AI provider's base_url goes through, and for the same
|
||||
// reason: this is an address a user typed, and the server is about to fetch it
|
||||
// on a schedule of their choosing.
|
||||
if reason := safedial.CheckURL(raw); reason != "" {
|
||||
return "", errors.New(reason)
|
||||
}
|
||||
return raw, nil
|
||||
}
|
||||
|
||||
// icsClient is the client a subscribed calendar is fetched with.
|
||||
//
|
||||
// safedial, checked on the *resolved* address at connect time and on every
|
||||
// redirect hop — because a name that passed the check when it was pasted can
|
||||
// answer 127.0.0.1 when the sweep runs at three in the morning.
|
||||
//
|
||||
// The field exists so a test can put an ordinary client here: httptest listens on
|
||||
// loopback, which is exactly what this refuses, so a test of the *format* cannot
|
||||
// otherwise reach its own server. It is never set outside a test, and the wall it
|
||||
// bypasses is tested where it belongs — `normaliseCalendarURL` refuses a private
|
||||
// address where it is written down, and `internal/push` proves the connect-time
|
||||
// half.
|
||||
func (s *Server) icsClient() *http.Client {
|
||||
if s.icsFetch != nil {
|
||||
return s.icsFetch
|
||||
}
|
||||
return safedial.Client(30 * time.Second)
|
||||
}
|
||||
|
||||
// fetchICS reads a subscribed calendar.
|
||||
//
|
||||
// safedial, not a plain client — checked on the resolved address at connect time
|
||||
// and on every redirect hop, because a name that passed the check when it was
|
||||
// pasted can answer 127.0.0.1 when the sweep runs at three in the morning.
|
||||
func (s *Server) fetchICS(ctx context.Context, url string) (ics.Subscribed, error) {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
|
||||
if err != nil {
|
||||
return ics.Subscribed{}, err
|
||||
}
|
||||
req.Header.Set("Accept", "text/calendar, text/plain;q=0.9, */*;q=0.5")
|
||||
req.Header.Set("User-Agent", "verdande/"+Version)
|
||||
|
||||
resp, err := s.icsClient().Do(req)
|
||||
if err != nil {
|
||||
return ics.Subscribed{}, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode >= 300 {
|
||||
return ics.Subscribed{}, fmt.Errorf("%s said %s", resp.Request.URL.Host, resp.Status)
|
||||
}
|
||||
|
||||
body, err := io.ReadAll(io.LimitReader(resp.Body, maxICSBytes+1))
|
||||
if err != nil {
|
||||
return ics.Subscribed{}, err
|
||||
}
|
||||
if len(body) > maxICSBytes {
|
||||
return ics.Subscribed{}, errors.New("that calendar is too large")
|
||||
}
|
||||
return ics.ParseCalendar(string(body)), nil
|
||||
}
|
||||
|
||||
// syncSubscriptions refreshes every calendar this person subscribes to by address.
|
||||
//
|
||||
// One failing address does not stop the others. A subscription is somebody else's
|
||||
// server, and the ordinary state of somebody else's server is that it is fine
|
||||
// until one morning it is not — a holiday feed that 404s must not take the
|
||||
// football club's fixtures down with it.
|
||||
func (s *Server) syncSubscriptions(ctx context.Context, user *store.User) (int, error) {
|
||||
subs, err := s.db.Subscriptions(ctx, user.ID)
|
||||
if err != nil || len(subs) == 0 {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
from, to := calendarWindow(time.Now())
|
||||
written := 0
|
||||
var failures []string
|
||||
|
||||
for _, sub := range subs {
|
||||
if ctx.Err() != nil {
|
||||
return written, ctx.Err()
|
||||
}
|
||||
unlock := s.lockSync("calendar:" + sub.ID)
|
||||
|
||||
calendarID, err := s.db.SubscribedCalendarID(ctx, sub.ID)
|
||||
if err != nil {
|
||||
unlock()
|
||||
failures = append(failures, sub.URL)
|
||||
continue
|
||||
}
|
||||
|
||||
cal, err := s.fetchICS(ctx, sub.URL)
|
||||
if err != nil {
|
||||
unlock()
|
||||
s.log.Warn("subscribed calendar", "err", err, "url", sub.URL, "user", user.ID)
|
||||
failures = append(failures, sub.URL)
|
||||
continue
|
||||
}
|
||||
|
||||
// The name the file calls itself, once. A subscription is added under its
|
||||
// address because that is all there is to go on before it has been fetched.
|
||||
if err := s.db.RenameSubscribedCalendar(ctx, sub.ID, cal.Name); err != nil {
|
||||
s.log.Warn("name subscribed calendar", "err", err)
|
||||
}
|
||||
|
||||
rows := make([]store.CalendarEvent, 0, len(cal.Events))
|
||||
for _, e := range cal.Events {
|
||||
// Outside the window is not stored. The grid asks about a window, the
|
||||
// sweep replaces a window, and a decade of holidays in one file would
|
||||
// otherwise be a decade of rows rewritten every fifteen minutes.
|
||||
if e.EndDay < from || e.StartDay > to {
|
||||
continue
|
||||
}
|
||||
rows = append(rows, store.CalendarEvent{
|
||||
RemoteID: e.UID, Summary: e.Summary, Location: e.Location, URL: e.URL,
|
||||
StartsAt: e.StartsAt, EndsAt: e.EndsAt,
|
||||
StartDay: e.StartDay, EndDay: e.EndDay, AllDay: e.AllDay,
|
||||
})
|
||||
}
|
||||
if err := s.db.ReplaceCalendarEvents(ctx, calendarID, from, to, rows); err != nil {
|
||||
unlock()
|
||||
return written, err
|
||||
}
|
||||
if err := s.db.MarkCalendarSynced(ctx, sub.ID, time.Now()); err != nil {
|
||||
s.log.Warn("mark subscription synced", "err", err)
|
||||
}
|
||||
written += len(rows)
|
||||
unlock()
|
||||
}
|
||||
|
||||
if len(failures) == len(subs) {
|
||||
return written, fmt.Errorf("none of the %d subscribed calendars could be read", len(subs))
|
||||
}
|
||||
return written, nil
|
||||
}
|
||||
|
||||
// handleSubscribeCalendar adds a calendar somebody only has an address to.
|
||||
//
|
||||
// Fetched once, here, before it is stored. An address that cannot be read is a
|
||||
// typo far more often than it is a server that happens to be down, and telling
|
||||
// somebody now — while they are looking at the field they just pasted into — is
|
||||
// worth the two seconds. A subscription that silently never fills is the same
|
||||
// silence the notifications had.
|
||||
func (s *Server) handleSubscribeCalendar(w http.ResponseWriter, r *http.Request) {
|
||||
var req struct {
|
||||
URL string `json:"url"`
|
||||
Name string `json:"name"`
|
||||
}
|
||||
if err := decodeJSON(w, r, &req); err != nil {
|
||||
return
|
||||
}
|
||||
|
||||
url, err := normaliseCalendarURL(req.URL)
|
||||
if err != nil {
|
||||
writeFieldErrors(w, map[string]string{"url": err.Error()})
|
||||
return
|
||||
}
|
||||
|
||||
user := userFrom(r.Context())
|
||||
ctx, cancel := context.WithTimeout(r.Context(), 20*time.Second)
|
||||
defer cancel()
|
||||
|
||||
cal, err := s.fetchICS(ctx, url)
|
||||
if err != nil {
|
||||
s.log.Warn("subscribe calendar", "err", err, "user", user.ID)
|
||||
writeFieldErrors(w, map[string]string{"url": "that address could not be read"})
|
||||
return
|
||||
}
|
||||
|
||||
name := strings.TrimSpace(req.Name)
|
||||
if name == "" {
|
||||
name = strings.TrimSpace(cal.Name)
|
||||
}
|
||||
if name == "" {
|
||||
name = url
|
||||
}
|
||||
|
||||
account, err := s.db.SubscribeCalendar(r.Context(), user.ID, url, name)
|
||||
if err != nil {
|
||||
// The unique index is the one refusal worth naming: subscribing twice to
|
||||
// the same address is the same calendar fetched twice, with no way to say
|
||||
// which one counts.
|
||||
if errors.Is(err, store.ErrDuplicate) {
|
||||
writeFieldErrors(w, map[string]string{"url": "that calendar is already subscribed"})
|
||||
return
|
||||
}
|
||||
s.storeError(w, r, "subscribe calendar", err)
|
||||
return
|
||||
}
|
||||
|
||||
// The events it already fetched, rather than making the person wait fifteen
|
||||
// minutes for the sweep to prove it worked.
|
||||
if _, err := s.syncSubscriptions(r.Context(), user); err != nil {
|
||||
s.log.Warn("first subscription sync", "err", err)
|
||||
}
|
||||
writeJSON(w, http.StatusCreated, account)
|
||||
}
|
||||
|
||||
func (s *Server) handleUnsubscribeCalendar(w http.ResponseWriter, r *http.Request) {
|
||||
user := userFrom(r.Context())
|
||||
if err := s.db.DeleteCalendarAccount(r.Context(), user.ID, chi.URLParam(r, "id")); err != nil {
|
||||
s.storeError(w, r, "unsubscribe calendar", err)
|
||||
return
|
||||
}
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
}
|
||||
@@ -0,0 +1,471 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/kristianwind/verdande/internal/config"
|
||||
"github.com/kristianwind/verdande/internal/store"
|
||||
)
|
||||
|
||||
// fakeCalendarGoogle stands in for the parts of Google this feature touches: the
|
||||
// token endpoint, the calendar list and one calendar's events.
|
||||
//
|
||||
// There is no other way to exercise any of it. The real thing needs a registered
|
||||
// OAuth client, an account and somebody's consent, and none of the three can be had
|
||||
// inside a test — so the shapes Google returns are written down and the behaviour
|
||||
// is checked against them, the way internal/imap is checked against go-imap's own
|
||||
// in-process server.
|
||||
type fakeCalendarGoogle struct {
|
||||
*httptest.Server
|
||||
// refreshes counts token refreshes, so a test can prove one is not made for a
|
||||
// token that has not expired.
|
||||
refreshes int
|
||||
// refuse makes every API call answer 401, which is what a revoked grant does.
|
||||
refuse bool
|
||||
events map[string][]map[string]any
|
||||
}
|
||||
|
||||
func newFakeCalendarGoogle(t *testing.T) *fakeCalendarGoogle {
|
||||
t.Helper()
|
||||
g := &fakeCalendarGoogle{events: map[string][]map[string]any{}}
|
||||
g.Server = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
switch {
|
||||
case r.URL.Path == "/token":
|
||||
g.refreshes++
|
||||
_ = json.NewEncoder(w).Encode(map[string]any{
|
||||
"access_token": "ya29.fresh", "refresh_token": "1//0-refresh",
|
||||
"expires_in": 3600,
|
||||
})
|
||||
case g.refuse:
|
||||
http.Error(w, `{"error":"invalid_credentials"}`, http.StatusUnauthorized)
|
||||
case strings.HasPrefix(r.URL.Path, "/users/me/calendarList"):
|
||||
_ = json.NewEncoder(w).Encode(map[string]any{"items": []map[string]any{
|
||||
{"id": "[email protected]", "summary": "[email protected]", "primary": true,
|
||||
"accessRole": "owner", "backgroundColor": "#3f51b5",
|
||||
"timeZone": "Europe/Copenhagen"},
|
||||
{"id": "fam", "summary": "Familien", "accessRole": "reader",
|
||||
"backgroundColor": "#0b8043"},
|
||||
}})
|
||||
case strings.HasPrefix(r.URL.Path, "/calendars/"):
|
||||
id := strings.TrimSuffix(strings.TrimPrefix(r.URL.Path, "/calendars/"), "/events")
|
||||
items := g.events[id]
|
||||
if items == nil {
|
||||
items = []map[string]any{}
|
||||
}
|
||||
_ = json.NewEncoder(w).Encode(map[string]any{"items": items})
|
||||
default:
|
||||
http.Error(w, "unexpected "+r.URL.Path, http.StatusNotFound)
|
||||
}
|
||||
}))
|
||||
t.Cleanup(g.Close)
|
||||
return g
|
||||
}
|
||||
|
||||
// connectedCalendar returns a server with a calendar account already connected —
|
||||
// the state the OAuth callback leaves behind, without the consent screen nothing
|
||||
// can automate.
|
||||
func connectedCalendar(t *testing.T, g *fakeCalendarGoogle, tune func(*config.Config)) (*testServer, *store.User) {
|
||||
t.Helper()
|
||||
ts := newTestServerWith(t, func(cfg *config.Config) {
|
||||
cfg.GmailClientID = "test-client"
|
||||
cfg.GmailClientSecret = "test-secret"
|
||||
cfg.GoogleTokenURL = g.URL + "/token"
|
||||
cfg.CalendarAPIURL = g.URL
|
||||
if tune != nil {
|
||||
tune(cfg)
|
||||
}
|
||||
})
|
||||
ts.bootstrap(t)
|
||||
|
||||
user, err := ts.db.UserByEmail(t.Context(), "[email protected]")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
account := &store.CalendarAccount{
|
||||
UserID: user.ID, Provider: "google", RefreshToken: "1//0-refresh",
|
||||
}
|
||||
if err := ts.db.SaveCalendarAccount(t.Context(), account); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return ts, user
|
||||
}
|
||||
|
||||
// The scope on the consent screen is what somebody agrees to. It has to be the
|
||||
// read-only one, and the redirect has to be the calendar's own — sharing Gmail's
|
||||
// would put "which token store do I write" behind a lookup.
|
||||
func TestAuthorizeAsksForReadOnlyCalendar(t *testing.T) {
|
||||
g := newFakeCalendarGoogle(t)
|
||||
ts, _ := connectedCalendar(t, g, nil)
|
||||
|
||||
resp, body := ts.do(t, "POST", "/api/v1/calendar/authorize", nil)
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
t.Fatalf("status %d, body %v", resp.StatusCode, body)
|
||||
}
|
||||
url, _ := body["url"].(string)
|
||||
for _, want := range []string{
|
||||
"scope=" + "https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcalendar.readonly",
|
||||
"%2Foauth%2Fcalendar%2Fcallback",
|
||||
"access_type=offline",
|
||||
"prompt=consent",
|
||||
} {
|
||||
if !strings.Contains(url, want) {
|
||||
t.Errorf("the authorisation URL is missing %q:\n%s", want, url)
|
||||
}
|
||||
}
|
||||
if strings.Contains(url, "calendar.events&") || strings.Contains(url, "auth%2Fcalendar&") {
|
||||
t.Errorf("a writing scope was asked for:\n%s", url)
|
||||
}
|
||||
}
|
||||
|
||||
// Without a registered client there is nothing to connect through, and the answer
|
||||
// has to say that rather than "that already exists".
|
||||
func TestAuthorizeWithoutAClientSaysSo(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
ts.bootstrap(t)
|
||||
|
||||
resp, body := ts.do(t, "POST", "/api/v1/calendar/authorize", nil)
|
||||
if resp.StatusCode != http.StatusConflict {
|
||||
t.Fatalf("status %d, want 409", resp.StatusCode)
|
||||
}
|
||||
if body["code"] != CodeGmailNotConfigured {
|
||||
t.Errorf("code = %v, want %q", body["code"], CodeGmailNotConfigured)
|
||||
}
|
||||
}
|
||||
|
||||
// The whole path, minus the consent screen: sync fetches the list, refreshes the
|
||||
// events of what was chosen, and the grid endpoint answers with them.
|
||||
func TestSyncFetchesTheChosenCalendarsAndTheGridReadsThem(t *testing.T) {
|
||||
g := newFakeCalendarGoogle(t)
|
||||
g.events["fam"] = []map[string]any{
|
||||
{"id": "e1", "summary": "Bestyrelsesmøde", "htmlLink": "https://calendar.google.com/e1",
|
||||
"start": map[string]string{"dateTime": "2026-08-20T14:00:00+02:00"},
|
||||
"end": map[string]string{"dateTime": "2026-08-20T15:30:00+02:00"}},
|
||||
}
|
||||
g.events["[email protected]"] = []map[string]any{
|
||||
{"id": "e2", "summary": "Skal ikke med",
|
||||
"start": map[string]string{"date": "2026-08-21"},
|
||||
"end": map[string]string{"date": "2026-08-22"}},
|
||||
}
|
||||
ts, user := connectedCalendar(t, g, nil)
|
||||
|
||||
// The list arrives on the first look at the settings page, which is what
|
||||
// somebody would be doing at this point.
|
||||
resp, body := ts.do(t, "GET", "/api/v1/calendar", nil)
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
t.Fatalf("status %d, body %v", resp.StatusCode, body)
|
||||
}
|
||||
if body["connected"] != true {
|
||||
t.Fatalf("not reported as connected: %v", body)
|
||||
}
|
||||
if body["read_only"] != true {
|
||||
t.Error("the connection did not say it is read-only; an interface that looks " +
|
||||
"editable is one somebody will try to edit")
|
||||
}
|
||||
// The primary calendar's id is the address, which is how the account names
|
||||
// itself without asking for a profile scope as well.
|
||||
if body["account"] != "[email protected]" {
|
||||
t.Errorf("account = %v", body["account"])
|
||||
}
|
||||
|
||||
list, _ := body["calendars"].([]any)
|
||||
if len(list) != 2 {
|
||||
t.Fatalf("got %d calendars, want 2: %v", len(list), body["calendars"])
|
||||
}
|
||||
var famID string
|
||||
for _, raw := range list {
|
||||
c := raw.(map[string]any)
|
||||
if c["shown"] == true {
|
||||
t.Errorf("%v was shown without anybody choosing it", c["name"])
|
||||
}
|
||||
if c["remote_id"] == "fam" {
|
||||
famID = c["id"].(string)
|
||||
}
|
||||
}
|
||||
|
||||
// Nothing is chosen, so a sync writes nothing: an account with a dozen shared
|
||||
// calendars must not fill somebody's grid the moment it is connected.
|
||||
resp, body = ts.do(t, "POST", "/api/v1/calendar/sync", nil)
|
||||
if resp.StatusCode != http.StatusOK || body["events"] != float64(0) {
|
||||
t.Fatalf("an unchosen calendar was fetched: %d %v", resp.StatusCode, body)
|
||||
}
|
||||
|
||||
resp, _ = ts.do(t, "PUT", "/api/v1/calendar/calendars",
|
||||
map[string]any{"shown": []string{famID}})
|
||||
if resp.StatusCode != http.StatusNoContent {
|
||||
t.Fatalf("choosing a calendar: status %d", resp.StatusCode)
|
||||
}
|
||||
|
||||
resp, body = ts.do(t, "POST", "/api/v1/calendar/sync", nil)
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
t.Fatalf("sync: status %d, body %v", resp.StatusCode, body)
|
||||
}
|
||||
if body["events"] != float64(1) {
|
||||
t.Fatalf("events = %v, want 1 — only the chosen calendar", body["events"])
|
||||
}
|
||||
|
||||
resp, body = ts.do(t, "GET",
|
||||
"/api/v1/calendar/events?from=2026-08-01&to=2026-08-31", nil)
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
t.Fatalf("events: status %d, body %v", resp.StatusCode, body)
|
||||
}
|
||||
events, _ := body["events"].([]any)
|
||||
if len(events) != 1 {
|
||||
t.Fatalf("got %d events, want 1: %v", len(events), body["events"])
|
||||
}
|
||||
event := events[0].(map[string]any)
|
||||
if event["summary"] != "Bestyrelsesmøde" || event["start_day"] != "2026-08-20" {
|
||||
t.Errorf("the event came back as %v", event)
|
||||
}
|
||||
// The calendar's colour rides along, or the chip has to ask for it per row.
|
||||
if event["colour"] != "#0b8043" {
|
||||
t.Errorf("colour = %v; two calendars in one grid are told apart by it", event["colour"])
|
||||
}
|
||||
// The window verdande holds is stated, so a grid paged past it can say so
|
||||
// rather than showing an empty month it cannot explain.
|
||||
if body["from"] == "" || body["to"] == "" {
|
||||
t.Error("the answer does not say which window it covers")
|
||||
}
|
||||
|
||||
// And it is one person's calendar. Somebody else on the same instance sees
|
||||
// nothing of it.
|
||||
other := ts.newUser(t, "[email protected]", "Andreas")
|
||||
_, body = other.do(t, "GET", "/api/v1/calendar/events?from=2026-08-01&to=2026-08-31", nil)
|
||||
if events, _ := body["events"].([]any); len(events) != 0 {
|
||||
t.Errorf("somebody else's calendar was visible: %v", body["events"])
|
||||
}
|
||||
_ = user
|
||||
}
|
||||
|
||||
// A token that has not expired must not be traded in for another. Refreshing on
|
||||
// every run burns the quota and makes the failure mode of a bad refresh token
|
||||
// arrive on every sweep rather than once an hour.
|
||||
func TestAFreshTokenIsNotRefreshed(t *testing.T) {
|
||||
g := newFakeCalendarGoogle(t)
|
||||
ts, user := connectedCalendar(t, g, nil)
|
||||
|
||||
account, err := ts.db.CalendarAccountFor(t.Context(), user.ID, "google")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
account.AccessToken = "ya29.still-good"
|
||||
account.ExpiresAt = time.Now().Add(time.Hour)
|
||||
if err := ts.db.SaveCalendarAccount(t.Context(), account); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if resp, body := ts.do(t, "POST", "/api/v1/calendar/sync", nil); resp.StatusCode != http.StatusOK {
|
||||
t.Fatalf("sync: status %d, body %v", resp.StatusCode, body)
|
||||
}
|
||||
if g.refreshes != 0 {
|
||||
t.Errorf("the token was refreshed %d times while it was still good", g.refreshes)
|
||||
}
|
||||
|
||||
// And an expired one is.
|
||||
account.ExpiresAt = time.Now().Add(-time.Minute)
|
||||
if err := ts.db.SaveCalendarAccount(t.Context(), account); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if resp, body := ts.do(t, "POST", "/api/v1/calendar/sync", nil); resp.StatusCode != http.StatusOK {
|
||||
t.Fatalf("sync: status %d, body %v", resp.StatusCode, body)
|
||||
}
|
||||
if g.refreshes != 1 {
|
||||
t.Errorf("an expired token was refreshed %d times, want 1", g.refreshes)
|
||||
}
|
||||
}
|
||||
|
||||
// Access revoked at Google's end is a connection that has ended, not a failure to
|
||||
// retry. Polling it every fifteen minutes for ever is a week of log lines and a
|
||||
// connection the person believes they removed.
|
||||
func TestRevokedAccessDisconnects(t *testing.T) {
|
||||
g := newFakeCalendarGoogle(t)
|
||||
ts, user := connectedCalendar(t, g, nil)
|
||||
|
||||
account, _ := ts.db.CalendarAccountFor(t.Context(), user.ID, "google")
|
||||
account.AccessToken = "ya29.revoked"
|
||||
account.ExpiresAt = time.Now().Add(time.Hour)
|
||||
if err := ts.db.SaveCalendarAccount(t.Context(), account); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
g.refuse = true
|
||||
|
||||
resp, body := ts.do(t, "POST", "/api/v1/calendar/sync", nil)
|
||||
if resp.StatusCode != StatusUpstreamRefused {
|
||||
t.Fatalf("status %d, want %d — Google saying no is not this server breaking",
|
||||
resp.StatusCode, StatusUpstreamRefused)
|
||||
}
|
||||
if body["code"] != CodeCalendarFailed {
|
||||
t.Errorf("code = %v, want %q — `internal_error` is shown as a generic sentence, "+
|
||||
"which throws the diagnosis away", body["code"], CodeCalendarFailed)
|
||||
}
|
||||
|
||||
gone, err := ts.db.CalendarAccountFor(t.Context(), user.ID, "google")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if gone != nil {
|
||||
t.Error("the connection is still there after Google refused the token")
|
||||
}
|
||||
|
||||
// And the failure is in the error log, which is the point of having one: a
|
||||
// sync that has been failing for a week is invisible otherwise.
|
||||
_, logged := ts.do(t, "GET", "/api/v1/errors", nil)
|
||||
rows, _ := logged["errors"].([]any)
|
||||
if len(rows) == 0 {
|
||||
t.Fatal("nothing was recorded")
|
||||
}
|
||||
if row := rows[0].(map[string]any); row["what"] != "sync calendar" {
|
||||
t.Errorf("what = %v, want \"sync calendar\"", row["what"])
|
||||
}
|
||||
}
|
||||
|
||||
// Disconnecting takes the copy with it. None of it is the person's own work, and
|
||||
// a calendar left behind after the account is gone is a frozen lie about a day.
|
||||
func TestDisconnectingRemovesTheEvents(t *testing.T) {
|
||||
g := newFakeCalendarGoogle(t)
|
||||
g.events["fam"] = []map[string]any{
|
||||
{"id": "e1", "summary": "Møde",
|
||||
"start": map[string]string{"date": "2026-08-20"},
|
||||
"end": map[string]string{"date": "2026-08-21"}},
|
||||
}
|
||||
ts, user := connectedCalendar(t, g, nil)
|
||||
|
||||
ts.do(t, "GET", "/api/v1/calendar", nil)
|
||||
account, _ := ts.db.CalendarAccountFor(t.Context(), user.ID, "google")
|
||||
calendars, _ := ts.db.Calendars(t.Context(), account.ID)
|
||||
var famID string
|
||||
for _, c := range calendars {
|
||||
if c.RemoteID == "fam" {
|
||||
famID = c.ID
|
||||
}
|
||||
}
|
||||
ts.do(t, "PUT", "/api/v1/calendar/calendars", map[string]any{"shown": []string{famID}})
|
||||
ts.do(t, "POST", "/api/v1/calendar/sync", nil)
|
||||
|
||||
if resp, _ := ts.do(t, "DELETE", "/api/v1/calendar", nil); resp.StatusCode != http.StatusNoContent {
|
||||
t.Fatalf("disconnect: status %d", resp.StatusCode)
|
||||
}
|
||||
|
||||
_, body := ts.do(t, "GET", "/api/v1/calendar/events?from=2026-08-01&to=2026-08-31", nil)
|
||||
if events, _ := body["events"].([]any); len(events) != 0 {
|
||||
t.Errorf("events survived the disconnect: %v", body["events"])
|
||||
}
|
||||
_, body = ts.do(t, "GET", "/api/v1/calendar", nil)
|
||||
if body["connected"] != false {
|
||||
t.Error("still reported as connected")
|
||||
}
|
||||
}
|
||||
|
||||
// A range that is not a date has to be refused, not guessed at. An empty `from`
|
||||
// compared as a string matches everything before it, which is a grid quietly
|
||||
// answering a question nobody asked.
|
||||
func TestEventsRefuseARangeThatIsNotADate(t *testing.T) {
|
||||
g := newFakeCalendarGoogle(t)
|
||||
ts, _ := connectedCalendar(t, g, nil)
|
||||
|
||||
for _, query := range []string{
|
||||
"", "?from=2026-08-01", "?from=&to=2026-08-31",
|
||||
"?from=i+morgen&to=2026-08-31", "?from=2026-08-01&to=2026-13-40",
|
||||
} {
|
||||
resp, body := ts.do(t, "GET", "/api/v1/calendar/events"+query, nil)
|
||||
if resp.StatusCode != http.StatusUnprocessableEntity {
|
||||
t.Errorf("%q: status %d, want 422 (%v)", query, resp.StatusCode, body)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A person with no calendar connected still opens the view. It has to answer, and
|
||||
// answer emptily, rather than fail — the grid is also verdande's own tasks, and
|
||||
// they must not disappear because Google is not set up.
|
||||
func TestTheGridAnswersWithNoCalendarConnected(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
ts.bootstrap(t)
|
||||
|
||||
resp, body := ts.do(t, "GET", "/api/v1/calendar/events?from=2026-08-01&to=2026-08-31", nil)
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
t.Fatalf("status %d, body %v", resp.StatusCode, body)
|
||||
}
|
||||
if events, ok := body["events"].([]any); !ok || len(events) != 0 {
|
||||
t.Errorf("events = %v, want an empty list rather than null", body["events"])
|
||||
}
|
||||
|
||||
resp, body = ts.do(t, "GET", "/api/v1/calendar", nil)
|
||||
if resp.StatusCode != http.StatusOK || body["connected"] != false {
|
||||
t.Fatalf("status %d, body %v", resp.StatusCode, body)
|
||||
}
|
||||
if body["has_client"] != false {
|
||||
t.Error("an instance with no OAuth registration claimed to have one; the page " +
|
||||
"would offer a button that answers 409")
|
||||
}
|
||||
// Spelled out even when nothing is connected: it is what has to be registered
|
||||
// with Google, and it is the single most likely thing to be wrong.
|
||||
if uri, _ := body["redirect_uri"].(string); !strings.HasSuffix(uri, "/oauth/calendar/callback") {
|
||||
t.Errorf("redirect_uri = %q", uri)
|
||||
}
|
||||
}
|
||||
|
||||
// A slow Google must not hold the request open until something in front of this
|
||||
// server gives up on it and answers with its own HTML.
|
||||
func TestASlowCalendarDoesNotHangTheRequest(t *testing.T) {
|
||||
blocked := make(chan struct{})
|
||||
defer close(blocked)
|
||||
slow := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
select {
|
||||
case <-blocked:
|
||||
case <-r.Context().Done():
|
||||
}
|
||||
}))
|
||||
defer slow.Close()
|
||||
|
||||
ts := newTestServerWith(t, func(cfg *config.Config) {
|
||||
cfg.GmailClientID = "test-client"
|
||||
cfg.GmailClientSecret = "test-secret"
|
||||
cfg.GoogleTokenURL = slow.URL + "/token"
|
||||
cfg.CalendarAPIURL = slow.URL
|
||||
// A second rather than the real twenty-five: what is being tested is that a
|
||||
// budget is applied at all, and spending the production one on every run
|
||||
// would put half a minute into the suite to learn the same thing.
|
||||
cfg.CalendarSyncBudget = time.Second
|
||||
})
|
||||
ts.bootstrap(t)
|
||||
|
||||
user, err := ts.db.UserByEmail(t.Context(), "[email protected]")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := ts.db.SaveCalendarAccount(t.Context(), &store.CalendarAccount{
|
||||
UserID: user.ID, Provider: "google", RefreshToken: "r",
|
||||
AccessToken: "a", ExpiresAt: time.Now().Add(time.Hour),
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// Its own client, because the shared one gives up after ten seconds — which is
|
||||
// shorter than the budget being tested and would prove nothing either way.
|
||||
req, _ := http.NewRequest("POST", ts.URL+"/api/v1/calendar/sync", nil)
|
||||
req.Header.Set("Sec-Fetch-Site", "same-origin")
|
||||
for _, c := range ts.client.Jar.Cookies(mustParse(t, ts.URL)) {
|
||||
req.AddCookie(c)
|
||||
}
|
||||
|
||||
started := time.Now()
|
||||
resp, err := (&http.Client{Timeout: 90 * time.Second}).Do(req)
|
||||
if err != nil {
|
||||
t.Fatalf("the request never came back (%v after %s); the sync has no budget of its "+
|
||||
"own, and whatever sits in front of this server answers for it",
|
||||
err, time.Since(started).Round(time.Second))
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if took := time.Since(started); took > 10*time.Second {
|
||||
t.Errorf("the sync took %s; the budget is not being applied", took.Round(time.Second))
|
||||
}
|
||||
if resp.StatusCode >= 500 {
|
||||
t.Errorf("status %d — a slow Google is not this server breaking", resp.StatusCode)
|
||||
}
|
||||
}
|
||||
@@ -225,17 +225,32 @@ func (s *Server) handleUploadAttachment(w http.ResponseWriter, r *http.Request)
|
||||
|
||||
a := &store.Attachment{
|
||||
TaskID: taskID,
|
||||
CommentID: r.FormValue("comment_id"),
|
||||
Filename: safeFilename(header.Filename),
|
||||
MimeType: detectMime(header.Filename, header.Header.Get("Content-Type")),
|
||||
Size: size,
|
||||
Path: stored,
|
||||
UploadedBy: user.ID,
|
||||
}
|
||||
|
||||
// An attachment hangs on a task or on a comment, never both — the schema says
|
||||
// so, and the form may have supplied a comment id.
|
||||
if a.CommentID != "" {
|
||||
a.TaskID = ""
|
||||
//
|
||||
// The comment is resolved and checked against the task in the URL. It was
|
||||
// taken on trust, which meant the permission check above guarded the *task
|
||||
// somebody named* while the file landed on *whatever comment they asked for*.
|
||||
// A viewer on a shared project can list its comments, so she could read an id
|
||||
// there and then post to a task in her own project with that id attached: the
|
||||
// file appears under somebody else's comment, in a project she may only read,
|
||||
// and is served to everyone in it from this origin under a name she chose.
|
||||
// Somebody removed from a project could do the same with ids they kept.
|
||||
if id := r.FormValue("comment_id"); id != "" {
|
||||
owner, err := s.db.CommentTask(r.Context(), id)
|
||||
if err != nil || owner != taskID {
|
||||
os.Remove(filepath.Join(s.cfg.FilesDir(), stored))
|
||||
writeError(w, http.StatusNotFound, CodeNotFound, "not found")
|
||||
return
|
||||
}
|
||||
a.CommentID, a.TaskID = id, ""
|
||||
}
|
||||
|
||||
if err := s.db.CreateAttachment(r.Context(), a); err != nil {
|
||||
@@ -327,6 +342,30 @@ func (s *Server) reachAttachment(r *http.Request, attachmentID string) (canEdit
|
||||
return true, nil
|
||||
}
|
||||
|
||||
// A file inside a note follows the note, which follows its project — the same
|
||||
// chain everything else here follows. Without this a picture imported with a
|
||||
// note answered 404 while sitting in the middle of the text that named it.
|
||||
if a.NoteID != "" {
|
||||
n, err := s.db.Note(r.Context(), a.NoteID)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
if n == nil {
|
||||
return false, store.ErrNotFound
|
||||
}
|
||||
if n.ProjectID == "" {
|
||||
if n.CreatedBy != user.ID {
|
||||
return false, store.ErrNotFound
|
||||
}
|
||||
return true, nil
|
||||
}
|
||||
role, err := store.RequireProjectRole(r.Context(), s.db, n.ProjectID, user.ID, store.RoleViewer)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
return role.CanEdit(), nil
|
||||
}
|
||||
|
||||
taskID, err := s.db.AttachmentTask(r.Context(), attachmentID)
|
||||
if err != nil {
|
||||
return false, err
|
||||
@@ -370,12 +409,30 @@ func (s *Server) handleDownloadAttachment(w http.ResponseWriter, r *http.Request
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
// Everything is served as a download rather than rendered. An uploaded SVG or
|
||||
// HTML file displayed inline would run its own script on this origin, with the
|
||||
// session cookie attached — attachments are the one place a user supplies
|
||||
// content that another user opens.
|
||||
w.Header().Set("Content-Type", "application/octet-stream")
|
||||
w.Header().Set("Content-Disposition", "attachment; filename*=UTF-8''"+
|
||||
// Almost everything is served as a download rather than rendered. An uploaded
|
||||
// SVG or HTML file displayed inline would run its own script on this origin,
|
||||
// with the session cookie attached — attachments are the one place a user
|
||||
// supplies content that another user opens.
|
||||
//
|
||||
// The exception is a short list of raster images, and it is not a softening of
|
||||
// that rule but what makes a note work: a note carries its pictures in the text
|
||||
// as ``, and served as octet-stream every one of them is a broken image.
|
||||
// The whole Apple Notes import exists to bring those pictures along.
|
||||
//
|
||||
// Safe because it is an allowlist of formats that cannot carry script, sent as
|
||||
// the exact type with `nosniff` — so the browser must treat it as that image or
|
||||
// not at all, and can never be talked into reading it as HTML. SVG is
|
||||
// deliberately absent: it is a document, not a picture, and it is the one image
|
||||
// type that can run script.
|
||||
disposition := "attachment"
|
||||
contentType := "application/octet-stream"
|
||||
if inlineImage[a.MimeType] {
|
||||
disposition = "inline"
|
||||
contentType = a.MimeType
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", contentType)
|
||||
w.Header().Set("Content-Disposition", disposition+"; filename*=UTF-8''"+
|
||||
strings.ReplaceAll(mime.QEncoding.Encode("utf-8", a.Filename), " ", "%20"))
|
||||
w.Header().Set("X-Content-Type-Options", "nosniff")
|
||||
w.Header().Set("Content-Length", fmt.Sprint(a.Size))
|
||||
@@ -384,6 +441,24 @@ func (s *Server) handleDownloadAttachment(w http.ResponseWriter, r *http.Request
|
||||
http.ServeContent(w, r, a.Filename, a.CreatedAt, f)
|
||||
}
|
||||
|
||||
// inlineImage is what may be shown rather than downloaded.
|
||||
//
|
||||
// Raster formats only, and named one by one rather than matched on an "image/"
|
||||
// prefix: the prefix would let image/svg+xml through, which is a document that
|
||||
// can run script, and a list that grows by accident is how that happens.
|
||||
var inlineImage = map[string]bool{
|
||||
"image/png": true,
|
||||
"image/jpeg": true,
|
||||
"image/gif": true,
|
||||
"image/webp": true,
|
||||
"image/avif": true,
|
||||
// Sendes med, selv om ingen browser viser den endnu. Den er lige så ufarlig
|
||||
// som de andre, og en iPhone-note er fuld af dem — den dag Safari viser dem,
|
||||
// virker de, uden at nogen skal huske hvorfor de ikke gjorde.
|
||||
"image/heic": true,
|
||||
"image/heif": true,
|
||||
}
|
||||
|
||||
func (s *Server) handleDeleteAttachment(w http.ResponseWriter, r *http.Request) {
|
||||
attachmentID := chi.URLParam(r, "attachmentID")
|
||||
|
||||
|
||||
@@ -0,0 +1,66 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// Politikken skal stå i svaret, og den skal stadig være stram.
|
||||
//
|
||||
// Den er dét, der gjorde en XSS i noteeditoren til indsat HTML frem for kørt
|
||||
// script, dengang en `alt` slap uden om sin undslupning — og der var ingen prøve
|
||||
// på, at headeren overhovedet blev sendt. En sikkerhedsforanstaltning, ingen
|
||||
// prøve rører, er en, der kan forsvinde i en oprydning uden at nogen opdager det.
|
||||
func TestEveryResponseCarriesAStrictCSP(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
ts.bootstrap(t)
|
||||
|
||||
resp, _ := ts.do(t, "GET", "/api/v1/auth/me", nil)
|
||||
policy := resp.Header.Get("Content-Security-Policy")
|
||||
if policy == "" {
|
||||
t.Fatal("der er ingen Content-Security-Policy på svaret")
|
||||
}
|
||||
|
||||
// Målt på script-src alene, og ikke på hele politikken.
|
||||
//
|
||||
// `style-src 'unsafe-inline'` står der med vilje og er forklaret i csp.go:
|
||||
// Svelte skriver stilarter ind i dokumentet, og en indsat stil kan gøre en
|
||||
// side grim, ikke køre kode. Det er script-src, der afgør, om indsat HTML
|
||||
// bliver til et kørende script, og det er derfor den, prøven ser på.
|
||||
scriptSrc := ""
|
||||
for _, part := range strings.Split(policy, ";") {
|
||||
if strings.HasPrefix(strings.TrimSpace(part), "script-src") {
|
||||
scriptSrc = part
|
||||
}
|
||||
}
|
||||
if scriptSrc == "" {
|
||||
t.Fatal("der er ingen script-src i politikken")
|
||||
}
|
||||
for _, forbidden := range []string{"'unsafe-inline'", "'unsafe-eval'", "'unsafe-hashes'", "'strict-dynamic'"} {
|
||||
if strings.Contains(scriptSrc, forbidden) {
|
||||
t.Errorf("script-src indeholder %s: %q", forbidden, strings.TrimSpace(scriptSrc))
|
||||
}
|
||||
}
|
||||
|
||||
// Og de fire, der skal være der.
|
||||
for _, want := range []string{"script-src", "object-src 'none'", "base-uri 'none'", "frame-ancestors 'none'"} {
|
||||
if !strings.Contains(policy, want) {
|
||||
t.Errorf("politikken mangler %q", want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Et bilag må ikke kunne blive læst som et dokument.
|
||||
func TestAttachmentsAreNotServedAsDocuments(t *testing.T) {
|
||||
for _, mime := range []string{"image/svg+xml", "text/html", "application/xhtml+xml", "text/xml"} {
|
||||
if inlineImage[mime] {
|
||||
t.Errorf("%s står på listen over det, der vises inline", mime)
|
||||
}
|
||||
}
|
||||
// Og listen skal faktisk kunne noget, ellers virker billeder i noter ikke.
|
||||
for _, mime := range []string{"image/png", "image/jpeg"} {
|
||||
if !inlineImage[mime] {
|
||||
t.Errorf("%s mangler; billeder i noter kan ikke vises", mime)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -63,6 +63,11 @@ const (
|
||||
// diagnosis — a wrong app password, a host that is not listening, a folder that
|
||||
// is not there — and it says it in its own words.
|
||||
CodeMailboxFailed = "mailbox_failed"
|
||||
|
||||
// And for a calendar read from Google. Its own code rather than the Gmail one,
|
||||
// because the sentence the interface wraps it in names where it came from — and
|
||||
// "Gmail said no" about a calendar sends somebody to the wrong panel.
|
||||
CodeCalendarFailed = "calendar_failed"
|
||||
)
|
||||
|
||||
// errLastAdmin is returned to the caller that already wrote the response, so it
|
||||
|
||||
@@ -25,18 +25,16 @@ const defaultGmailSyncBudget = 25 * time.Second
|
||||
// to redeploy a container to paste a client id.
|
||||
// gmailClient is the API client, pointed wherever the config says Google is.
|
||||
func (s *Server) gmailClient(accessToken string) *gmail.Client {
|
||||
return gmail.NewClient(accessToken).At(gmail.Endpoints{
|
||||
Token: s.cfg.GmailTokenURL, API: s.cfg.GmailAPIURL,
|
||||
})
|
||||
return gmail.NewClient(accessToken).At(s.cfg.GmailAPIURL)
|
||||
}
|
||||
|
||||
func (s *Server) gmailConfig(ctx context.Context) gmail.Config {
|
||||
endpoints := gmail.Endpoints{Token: s.cfg.GmailTokenURL, API: s.cfg.GmailAPIURL}
|
||||
cfg := gmail.Config{
|
||||
ClientID: s.cfg.GmailClientID,
|
||||
ClientSecret: s.cfg.GmailClientSecret,
|
||||
RedirectURL: s.cfg.GmailRedirectURL(),
|
||||
Endpoints: endpoints,
|
||||
Scope: gmail.Scope,
|
||||
TokenURL: s.cfg.GoogleTokenURL,
|
||||
}
|
||||
if cfg.ClientID != "" && cfg.ClientSecret != "" {
|
||||
return cfg
|
||||
@@ -251,6 +249,17 @@ func (s *Server) handleGmailSyncNow(w http.ResponseWriter, r *http.Request) {
|
||||
//
|
||||
// Returns how many were made. Also used by the background job, which is why it
|
||||
// takes a context and a user rather than a request.
|
||||
// maxGmailSeen bounds the list of message ids already turned into tasks.
|
||||
//
|
||||
// Gmail is read-only here — the scope is `gmail.readonly`, deliberately, because
|
||||
// asking for less is what makes the consent screen something a person can agree
|
||||
// to — so verdande cannot unstar a message the way the IMAP side unflags one. The
|
||||
// record has to live on this side, and a record on this side has to be bounded.
|
||||
//
|
||||
// A count rather than the previous rule of "whatever Gmail returned this run",
|
||||
// which was not a bound but a way of forgetting.
|
||||
const maxGmailSeen = 2000
|
||||
|
||||
func (s *Server) SyncGmail(ctx context.Context, user *store.User) (int, error) {
|
||||
box, err := s.db.MailboxOfKind(ctx, user.ID, "gmail")
|
||||
if err != nil {
|
||||
@@ -260,7 +269,7 @@ func (s *Server) SyncGmail(ctx context.Context, user *store.User) (int, error) {
|
||||
// The same guard as the mailboxes read over IMAP, for the same reason: a
|
||||
// press of "Fetch now" during the sweep would otherwise make two runs from
|
||||
// one marker.
|
||||
unlock := s.lockMailbox(box.ID)
|
||||
unlock := s.lockSync(box.ID)
|
||||
defer unlock()
|
||||
if fresh, err := s.db.MailboxOfKind(ctx, user.ID, "gmail"); err == nil && fresh != nil {
|
||||
box = fresh
|
||||
@@ -362,14 +371,35 @@ func (s *Server) SyncGmail(ctx context.Context, user *store.User) (int, error) {
|
||||
}
|
||||
|
||||
if created > 0 {
|
||||
// Only the ids Gmail still returns are kept, so the list cannot grow
|
||||
// without bound as messages fall out of the 30-day window.
|
||||
keep := make([]string, 0, len(ids))
|
||||
// The ids this run saw, on top of the ones already known.
|
||||
//
|
||||
// It used to keep only the ids Gmail returned *this run*, to stop the list
|
||||
// growing without bound. That threw away every id that was not on the page
|
||||
// it happened to fetch — and the query is capped, so on an account with more
|
||||
// starred mail than one page, each run forgot the messages the last one had
|
||||
// remembered. They came back unseen, and became tasks again, and again. That
|
||||
// is the duplicate.
|
||||
//
|
||||
// Bounded by a count instead. Newest first, and a ceiling far above any
|
||||
// plausible page, so an id leaves the list because it is old rather than
|
||||
// because it was not in the last answer.
|
||||
keep := make([]string, 0, len(ids)+len(box.Seen))
|
||||
added := map[string]bool{}
|
||||
for _, id := range ids {
|
||||
if seen[id] {
|
||||
if seen[id] && !added[id] {
|
||||
keep = append(keep, id)
|
||||
added[id] = true
|
||||
}
|
||||
}
|
||||
for _, id := range box.Seen {
|
||||
if !added[id] {
|
||||
keep = append(keep, id)
|
||||
added[id] = true
|
||||
}
|
||||
}
|
||||
if len(keep) > maxGmailSeen {
|
||||
keep = keep[:maxGmailSeen]
|
||||
}
|
||||
box.Seen = keep
|
||||
box.LastSyncAt = time.Now()
|
||||
if err := s.db.SaveMailbox(ctx, box); err != nil {
|
||||
|
||||
@@ -29,7 +29,7 @@ func TestAGmailFailureSaysWhatGmailSaidAndIsRecorded(t *testing.T) {
|
||||
ts := newTestServerWith(t, func(cfg *config.Config) {
|
||||
cfg.GmailClientID = "test-client"
|
||||
cfg.GmailClientSecret = "test-secret"
|
||||
cfg.GmailTokenURL = google.URL + "/token"
|
||||
cfg.GoogleTokenURL = google.URL + "/token"
|
||||
cfg.GmailAPIURL = google.URL
|
||||
})
|
||||
ts.bootstrap(t)
|
||||
@@ -97,7 +97,7 @@ func TestASlowMailboxDoesNotHangTheRequest(t *testing.T) {
|
||||
ts := newTestServerWith(t, func(cfg *config.Config) {
|
||||
cfg.GmailClientID = "test-client"
|
||||
cfg.GmailClientSecret = "test-secret"
|
||||
cfg.GmailTokenURL = google.URL + "/token"
|
||||
cfg.GoogleTokenURL = google.URL + "/token"
|
||||
cfg.GmailAPIURL = google.URL
|
||||
// A second rather than the real twenty-five: what is being tested is that a
|
||||
// budget is applied at all, and spending the production one on every run
|
||||
|
||||
@@ -11,6 +11,7 @@ import (
|
||||
"github.com/kristianwind/verdande/internal/ai"
|
||||
"github.com/kristianwind/verdande/internal/auth"
|
||||
"github.com/kristianwind/verdande/internal/quickadd"
|
||||
"github.com/kristianwind/verdande/internal/safedial"
|
||||
"github.com/kristianwind/verdande/internal/store"
|
||||
)
|
||||
|
||||
@@ -214,6 +215,24 @@ func (s *Server) handleSetAISettings(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
|
||||
// The address is checked where it is typed, as well as when it is dialled.
|
||||
//
|
||||
// safedial refuses a private destination at connect time and is the actual
|
||||
// wall; this is so the person entering it hears about it now rather than
|
||||
// discovering it later as a failed summary. A scheme other than http or https
|
||||
// is refused outright — file:// and gopher:// are not providers.
|
||||
if url := strings.TrimSpace(req.BaseURL); url != "" {
|
||||
if !strings.HasPrefix(url, "http://") && !strings.HasPrefix(url, "https://") {
|
||||
writeFieldErrors(w, map[string]string{"base_url": "must start with http:// or https://"})
|
||||
return
|
||||
}
|
||||
if reason := safedial.CheckURL(url); reason != "" {
|
||||
writeFieldErrors(w, map[string]string{"base_url": reason})
|
||||
return
|
||||
}
|
||||
req.BaseURL = url
|
||||
}
|
||||
|
||||
existing, err := s.aiConfig(r, user.ID)
|
||||
if err != nil {
|
||||
s.internal(w, r, "ai settings", err)
|
||||
|
||||
@@ -2,13 +2,18 @@ package httpapi
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/url"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/kristianwind/verdande/internal/config"
|
||||
"github.com/kristianwind/verdande/internal/store"
|
||||
)
|
||||
|
||||
// --- MCP -----------------------------------------------------------------------------
|
||||
@@ -250,6 +255,116 @@ func (ts *testServer) apiToken(t *testing.T, userID string) string {
|
||||
return token
|
||||
}
|
||||
|
||||
// The promise of sharing has to hold at every door, not just the API's.
|
||||
//
|
||||
// TestViewerCannotMutateThroughTheAPI asserts a viewer cannot write, and it
|
||||
// asserts it against /api/v1. CalDAV is a second door into the same tasks, and it
|
||||
// authorised differently: it checked the caller's standing in the project *named
|
||||
// in the URL* and then took the task id from the URL as well. Nothing tied the
|
||||
// two together.
|
||||
//
|
||||
// So a viewer on somebody else's project could name a project of their own —
|
||||
// their Inbox, which everybody owns and everybody may edit — and put the other
|
||||
// project's task id in the filename. The project check passed on the Inbox, and
|
||||
// the write landed on a task in a project they may only read.
|
||||
func TestAViewerCannotWriteThroughCalDAV(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
ts.bootstrap(t)
|
||||
ts.newUser(t, "[email protected]", "Viewer")
|
||||
|
||||
_, project := ts.do(t, "POST", "/api/v1/projects", map[string]any{"name": "Delt"})
|
||||
projectID := project["id"].(string)
|
||||
_, task := ts.do(t, "POST", "/api/v1/tasks", map[string]any{
|
||||
"content": "en opgave", "project_id": projectID,
|
||||
})
|
||||
taskID := task["id"].(string)
|
||||
|
||||
if resp, body := ts.do(t, "POST", "/api/v1/projects/"+projectID+"/invites", map[string]any{
|
||||
"email": "[email protected]", "role": "viewer",
|
||||
}); resp.StatusCode != http.StatusOK {
|
||||
t.Fatalf("invite: %d %v", resp.StatusCode, body)
|
||||
}
|
||||
|
||||
them, err := ts.db.UserByEmail(t.Context(), "[email protected]")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
token := ts.apiToken(t, them.ID)
|
||||
|
||||
// Their own Inbox: a project they own and may edit, and the one every account
|
||||
// has. It is what makes the URL check pass.
|
||||
projects, err := ts.db.ListProjects(t.Context(), them.ID, false)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var inbox string
|
||||
for _, p := range projects {
|
||||
if p.IsInbox {
|
||||
inbox = p.ID
|
||||
}
|
||||
}
|
||||
if inbox == "" {
|
||||
t.Fatal("the viewer has no inbox")
|
||||
}
|
||||
|
||||
vtodo := "BEGIN:VCALENDAR\r\nVERSION:2.0\r\nBEGIN:VTODO\r\nUID:" + taskID +
|
||||
"\r\nSUMMARY:kapret\r\nSTATUS:COMPLETED\r\nEND:VTODO\r\nEND:VCALENDAR\r\n"
|
||||
|
||||
resp := ts.dav(t, "PUT", "/caldav/"+them.ID+"/"+inbox+"/"+taskID+".ics",
|
||||
token, them.Email, vtodo)
|
||||
if resp.StatusCode < 400 {
|
||||
t.Errorf("a viewer wrote through CalDAV: status %d", resp.StatusCode)
|
||||
}
|
||||
|
||||
// And the task is as its owner left it. The status alone is not the assertion:
|
||||
// the question is whether anything moved.
|
||||
_, after := ts.do(t, "GET", "/api/v1/tasks/"+taskID, nil)
|
||||
if after["content"] != "en opgave" {
|
||||
t.Errorf("the task was renamed by a viewer: %v", after["content"])
|
||||
}
|
||||
if after["completed"] != false {
|
||||
t.Error("the task was completed by a viewer")
|
||||
}
|
||||
}
|
||||
|
||||
// A task may only be addressed under the project it is actually in.
|
||||
//
|
||||
// The same hole seen from the other side: an editor on two projects could reach a
|
||||
// task in one of them through the other's URL. Nothing is escalated by it — they
|
||||
// may edit both — but it is the check that was missing, and a URL that lies about
|
||||
// where a resource lives is how the viewer case got in.
|
||||
func TestCalDAVRefusesATaskFromAnotherProject(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
ts.bootstrap(t)
|
||||
|
||||
user, err := ts.db.UserByEmail(t.Context(), "[email protected]")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
token := ts.apiToken(t, user.ID)
|
||||
|
||||
_, one := ts.do(t, "POST", "/api/v1/projects", map[string]any{"name": "Et"})
|
||||
_, two := ts.do(t, "POST", "/api/v1/projects", map[string]any{"name": "To"})
|
||||
_, task := ts.do(t, "POST", "/api/v1/tasks", map[string]any{
|
||||
"content": "hører til Et", "project_id": one["id"].(string),
|
||||
})
|
||||
taskID := task["id"].(string)
|
||||
|
||||
vtodo := "BEGIN:VCALENDAR\r\nVERSION:2.0\r\nBEGIN:VTODO\r\nUID:" + taskID +
|
||||
"\r\nSUMMARY:flyttet i smug\r\nEND:VTODO\r\nEND:VCALENDAR\r\n"
|
||||
|
||||
resp := ts.dav(t, "PUT", "/caldav/"+user.ID+"/"+two["id"].(string)+"/"+taskID+".ics",
|
||||
token, user.Email, vtodo)
|
||||
if resp.StatusCode < 400 {
|
||||
t.Errorf("a task was written under the wrong project: status %d", resp.StatusCode)
|
||||
}
|
||||
|
||||
_, after := ts.do(t, "GET", "/api/v1/tasks/"+taskID, nil)
|
||||
if after["content"] != "hører til Et" {
|
||||
t.Errorf("the task changed: %v", after["content"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestCalDAVRequiresCredentials(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
ts.bootstrap(t)
|
||||
@@ -562,6 +677,48 @@ func TestAIKeyIsNeverReturned(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// Connecting something else must not delete the AI provider.
|
||||
//
|
||||
// Reported from use as "the AI settings are not saved": they were saved, and then
|
||||
// taken. `user_settings` had `user_id` as its whole primary key, and the upsert
|
||||
// renamed `scope` on conflict — so the `ai`, `gmail` and `calendar` scopes shared
|
||||
// one row and each write deleted the other two. Every write returned 204.
|
||||
//
|
||||
// Through the handlers, because that is where it was seen. The shape is asserted
|
||||
// against the store as well: see TestOnePersonCanHaveSettingsForMoreThanOneThing.
|
||||
func TestTheAIProviderSurvivesConnectingSomethingElse(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
ts.bootstrap(t)
|
||||
|
||||
ts.do(t, "PUT", "/api/v1/ai/settings", map[string]any{
|
||||
"provider": "anthropic", "model": "claude-sonnet-5", "api_key": "sk-hemmelig-nøgle",
|
||||
})
|
||||
|
||||
user, err := ts.db.UserByEmail(t.Context(), "[email protected]")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// What the calendar and the mailbox handlers do when a handshake starts: write
|
||||
// their own scope for the same person.
|
||||
for _, scope := range []string{"calendar", "gmail"} {
|
||||
if err := ts.db.SetUserSettings(t.Context(), user.ID, scope,
|
||||
map[string]any{"pkce_state": "s-" + scope}); err != nil {
|
||||
t.Fatalf("save %s: %v", scope, err)
|
||||
}
|
||||
}
|
||||
|
||||
_, body := ts.do(t, "GET", "/api/v1/ai/settings", nil)
|
||||
if body["provider"] != "anthropic" {
|
||||
t.Errorf("the provider is %v after connecting two other things, want anthropic", body["provider"])
|
||||
}
|
||||
if body["model"] != "claude-sonnet-5" {
|
||||
t.Errorf("the model is %v", body["model"])
|
||||
}
|
||||
if body["has_key"] != true {
|
||||
t.Error("the key was taken with it")
|
||||
}
|
||||
}
|
||||
|
||||
func TestGmailSettings(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
ts.bootstrap(t)
|
||||
@@ -1316,3 +1473,172 @@ func TestTheMailAddressCanBeRotated(t *testing.T) {
|
||||
t.Error("the second rotation returned the same address")
|
||||
}
|
||||
}
|
||||
|
||||
// "Der kommer ingen notifikationer" må kunne besvares.
|
||||
//
|
||||
// Alt om det var usynligt: indstillingssiden sagde "til for denne enhed", så snart
|
||||
// der fandtes et abonnement, og en push, tjenesten afviste, gik i en Debug-linje,
|
||||
// som ikke skrives på en instans, der kører på standardniveau. Man kunne ikke
|
||||
// skelne "browseren tilmeldte sig aldrig" fra "tjenesten afviste den" fra "der er
|
||||
// ikke noget forfaldent endnu" — tre forskellige problemer med ét udseende.
|
||||
func TestATestNotificationSaysWhatHappenedToIt(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
ts.bootstrap(t)
|
||||
|
||||
// Ingen enheder: svaret skal sige nul frem for at lade som om det gik godt.
|
||||
resp, body := ts.do(t, "POST", "/api/v1/push/test", map[string]any{
|
||||
"title": "Prøve", "body": "kom den frem?",
|
||||
})
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
t.Fatalf("status %d, body %v", resp.StatusCode, body)
|
||||
}
|
||||
if body["subscriptions"] != float64(0) {
|
||||
t.Errorf("subscriptions = %v uden tilmeldte enheder", body["subscriptions"])
|
||||
}
|
||||
|
||||
// En enhed, der peger på en tjeneste, som ikke findes. Den skal komme tilbage
|
||||
// som et navngivent afslag og ikke som tavshed.
|
||||
user, err := ts.db.UserByEmail(t.Context(), "[email protected]")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := ts.db.SavePushSubscription(t.Context(), &store.PushSubscription{
|
||||
UserID: user.ID,
|
||||
Endpoint: "https://push.example.invalid/abc",
|
||||
P256dh: "BEl62iUYgUivxIkv69yViEuiBIa-Ib9-SkvMeAtA3LFgDzkrxZJjSgSnfckjBJuBkr3qBUYIHBQFLXYp5Nksh8U",
|
||||
Auth: "aUZmY2xDS2FQUXJJVmJIeA",
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
_, body = ts.do(t, "POST", "/api/v1/push/test", map[string]any{"title": "Prøve"})
|
||||
if body["subscriptions"] != float64(1) {
|
||||
t.Fatalf("subscriptions = %v", body["subscriptions"])
|
||||
}
|
||||
failed, _ := body["failed"].([]any)
|
||||
if len(failed) != 1 {
|
||||
t.Fatalf("failed har %d poster, want 1 — afslaget blev slugt", len(failed))
|
||||
}
|
||||
first, _ := failed[0].(map[string]any)
|
||||
if first["service"] != "push.example.invalid" {
|
||||
t.Errorf("service = %v", first["service"])
|
||||
}
|
||||
if first["reason"] == "" || first["reason"] == nil {
|
||||
t.Error("afslaget kom uden en grund")
|
||||
}
|
||||
// Og aldrig hele adressen: resten af den er den legitimation, tjenesten gav
|
||||
// enheden, og den hører hverken i et svar, en log eller et skærmbillede.
|
||||
if strings.Contains(fmt.Sprint(first), "/abc") {
|
||||
t.Errorf("endepunktet lækkede med ud: %v", first)
|
||||
}
|
||||
}
|
||||
|
||||
// Et abonnement på en kalender, man kun har en adresse til.
|
||||
//
|
||||
// Bedt om som "abonnering af flere kalendere (ics og webcal?)". Google-vejen er
|
||||
// OAuth og én konto pr. person; det her er en adresse, og man har flere. Det er
|
||||
// også den eneste vej for en privat konto, som en Internal-registrering afviser
|
||||
// med `org_internal`.
|
||||
//
|
||||
// Prøven går uden om handleren og ind i det, den kalder. Grunden er væggen:
|
||||
// `httptest` lytter på loopback, og safedial nægter — både når adressen skrives
|
||||
// ned og igen når den hentes. Det er rigtigt, og det skal ikke kunne slås fra fra
|
||||
// en prøve. Så adressereglerne prøves for sig i TestWebcalIsRewrittenToHTTPS, og
|
||||
// her prøves resten af vejen: hentning, format, navn, gitteret og afmeldingen.
|
||||
func TestSubscribingToACalendarByAddress(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
ts.bootstrap(t)
|
||||
|
||||
day := time.Now().AddDate(0, 0, 2).Format("20060102")
|
||||
feedBody := "BEGIN:VCALENDAR\r\nVERSION:2.0\r\nX-WR-CALNAME:Helligdage\r\n" +
|
||||
"BEGIN:VEVENT\r\nUID:[email protected]\r\n" +
|
||||
"DTSTART;VALUE=DATE:" + day + "\r\nDTEND;VALUE=DATE:" + day + "\r\n" +
|
||||
"SUMMARY:Grundlovsdag\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n"
|
||||
|
||||
var hits int
|
||||
feed := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
hits++
|
||||
w.Header().Set("Content-Type", "text/calendar")
|
||||
w.Write([]byte(feedBody))
|
||||
}))
|
||||
defer feed.Close()
|
||||
ts.api.icsFetch = &http.Client{Timeout: 10 * time.Second}
|
||||
|
||||
user, err := ts.db.UserByEmail(t.Context(), "[email protected]")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// Tilføjet under sin adresse, som er alt der er at gå efter, før den er hentet.
|
||||
if _, err := ts.db.SubscribeCalendar(t.Context(), user.ID, feed.URL, feed.URL); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := ts.api.syncSubscriptions(t.Context(), user); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if hits == 0 {
|
||||
t.Fatal("adressen blev aldrig hentet")
|
||||
}
|
||||
|
||||
// Navnet kommer fra filen. X-WR-CALNAME er det, alle udgivere bruger; der
|
||||
// findes ingen standardegenskab for det.
|
||||
_, status := ts.do(t, "GET", "/api/v1/calendar", nil)
|
||||
subs, _ := status["subscriptions"].([]any)
|
||||
if len(subs) != 1 {
|
||||
t.Fatalf("%d abonnementer i svaret", len(subs))
|
||||
}
|
||||
first, _ := subs[0].(map[string]any)
|
||||
if first["account"] != "Helligdage" {
|
||||
t.Errorf("navnet er %v, want Helligdage", first["account"])
|
||||
}
|
||||
|
||||
// Og begivenheden står i gitteret.
|
||||
from := time.Now().AddDate(0, 0, -1).Format("2006-01-02")
|
||||
to := time.Now().AddDate(0, 0, 7).Format("2006-01-02")
|
||||
_, events := ts.do(t, "GET", "/api/v1/calendar/events?from="+from+"&to="+to, nil)
|
||||
list, _ := events["events"].([]any)
|
||||
if len(list) != 1 {
|
||||
t.Fatalf("%d begivenheder efter abonnementet", len(list))
|
||||
}
|
||||
if e, _ := list[0].(map[string]any); e["summary"] != "Grundlovsdag" {
|
||||
t.Errorf("begivenheden er %v", e["summary"])
|
||||
}
|
||||
|
||||
// Den samme adresse to gange er den samme kalender hentet to gange, og der er
|
||||
// ingen måde at sige hvilken der gælder.
|
||||
if _, err := ts.db.SubscribeCalendar(t.Context(), user.ID, feed.URL, ""); !errors.Is(err, store.ErrDuplicate) {
|
||||
t.Errorf("den samme adresse blev accepteret to gange: %v", err)
|
||||
}
|
||||
|
||||
// Fjernet igen, og begivenhederne går med.
|
||||
resp, _ := ts.do(t, "DELETE", "/api/v1/calendar/subscriptions/"+first["id"].(string), nil)
|
||||
if resp.StatusCode != http.StatusNoContent {
|
||||
t.Fatalf("unsubscribe: status %d", resp.StatusCode)
|
||||
}
|
||||
_, events = ts.do(t, "GET", "/api/v1/calendar/events?from="+from+"&to="+to, nil)
|
||||
if list, _ := events["events"].([]any); len(list) != 0 {
|
||||
t.Errorf("%d begivenheder blev tilbage efter afmeldingen", len(list))
|
||||
}
|
||||
}
|
||||
|
||||
// webcal:// er https med et andet skema foran. Det er ikke en protokol, noget
|
||||
// taler — det er en konvention, der får et skrivebord til at give adressen til
|
||||
// kalenderprogrammet i stedet for til browseren, og alle udgivere skriver deres
|
||||
// links sådan. At afvise den ville være at afvise folk på en teknikalitet.
|
||||
func TestWebcalIsRewrittenToHTTPS(t *testing.T) {
|
||||
got, err := normaliseCalendarURL("webcal://example.dk/kalender.ics")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got != "https://example.dk/kalender.ics" {
|
||||
t.Errorf("blev til %q", got)
|
||||
}
|
||||
|
||||
// Og en adresse indad afvises, hvor den skrives — ikke først når den hentes.
|
||||
if _, err := normaliseCalendarURL("http://127.0.0.1:8080/x.ics"); err == nil {
|
||||
t.Error("en loopback-adresse blev godtaget")
|
||||
}
|
||||
if _, err := normaliseCalendarURL("file:///etc/passwd"); err == nil {
|
||||
t.Error("et skema, der ikke er http, blev godtaget")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -143,7 +143,7 @@ func (s *Server) SyncMailbox(ctx context.Context, user *store.User, m *store.Mai
|
||||
// One reader of this mailbox at a time. Held for the whole run, marker and all:
|
||||
// releasing before the marker is written would leave exactly the gap this is
|
||||
// here to close.
|
||||
unlock := s.lockMailbox(m.ID)
|
||||
unlock := s.lockSync(m.ID)
|
||||
defer unlock()
|
||||
|
||||
// Re-read under the lock. The row that came in may have been fetched before the
|
||||
@@ -175,6 +175,10 @@ func (s *Server) SyncMailbox(ctx context.Context, user *store.User, m *store.Mai
|
||||
|
||||
created := 0
|
||||
highest := m.LastUID
|
||||
// The uids of the mails that actually became tasks. Only those get unflagged,
|
||||
// and only after the task is committed: a flag removed from a mail whose task
|
||||
// was never written is a mail nobody will ever see again.
|
||||
done := make([]uint32, 0, len(messages))
|
||||
for _, msg := range messages {
|
||||
if ctx.Err() != nil {
|
||||
// What was made so far is already committed and counted, and the marker
|
||||
@@ -204,12 +208,27 @@ func (s *Server) SyncMailbox(ctx context.Context, user *store.User, m *store.Mai
|
||||
continue
|
||||
}
|
||||
created++
|
||||
done = append(done, msg.UID)
|
||||
if msg.UID > highest {
|
||||
highest = msg.UID
|
||||
}
|
||||
s.hub.Publish(projectID, "task.created", toTaskJSON(*task))
|
||||
}
|
||||
|
||||
// The flag comes off what became a task. This is what actually stops a mail
|
||||
// arriving twice; the marker below is now a second line rather than the only
|
||||
// one. See imap.Unflag for why the record belongs in the mailbox and not here.
|
||||
//
|
||||
// Outside the request's context, like the marker: a cancelled request that
|
||||
// already created tasks must still take the flags off them, or the next run
|
||||
// makes them all again.
|
||||
if err := client.Unflag(done); err != nil {
|
||||
// Not fatal, and not the caller's problem. A folder somebody opened
|
||||
// read-only, or a server that refuses the STORE, still gets the uid marker
|
||||
// underneath — worse, but not broken.
|
||||
s.log.Warn("could not unflag imported mail", "err", err, "mailbox", m.ID, "count", len(done))
|
||||
}
|
||||
|
||||
// A run that got all the way through moves the marker to the whole of what the
|
||||
// search considered, not merely to the last message it happened to read. That
|
||||
// is what stops the same mail becoming a task again on the next run when the
|
||||
@@ -231,8 +250,14 @@ func (s *Server) SyncMailbox(ctx context.Context, user *store.User, m *store.Mai
|
||||
return created, nil
|
||||
}
|
||||
|
||||
// lockMailbox serialises one mailbox against itself and returns the release.
|
||||
func (s *Server) lockMailbox(id string) func() {
|
||||
// lockSync serialises one thing that polls against itself and returns the release.
|
||||
//
|
||||
// A mailbox to begin with; a calendar connection now takes the same lock under a
|
||||
// key of its own. Two runs at once — somebody pressing "fetch now" while the sweep
|
||||
// is in the middle of the same thing — both start from the state as it was, and
|
||||
// the loser writes last. For a mailbox that made forty copies of one mail in a
|
||||
// second; for a calendar it would be a window deleted after it was refilled.
|
||||
func (s *Server) lockSync(id string) func() {
|
||||
value, _ := s.syncing.LoadOrStore(id, &sync.Mutex{})
|
||||
mu := value.(*sync.Mutex)
|
||||
mu.Lock()
|
||||
|
||||
@@ -101,6 +101,7 @@ func (s *Server) buildMCP() *mcp.Server {
|
||||
}, "task_id", "body"),
|
||||
}, s.mcpAddComment)
|
||||
|
||||
s.registerNoteTools(m)
|
||||
return m
|
||||
}
|
||||
|
||||
@@ -496,3 +497,168 @@ func clamp(value, fallback, max int) int {
|
||||
}
|
||||
return value
|
||||
}
|
||||
|
||||
// --- notes ---------------------------------------------------------------------
|
||||
|
||||
// registerNoteTools adds the note half of the MCP surface.
|
||||
//
|
||||
// Separate from the task tools for one reason worth stating: this is how a note
|
||||
// gets in from somewhere else. Apple Notes has no export worth the name, but a
|
||||
// model running on the machine can read Notes.app and write what it finds through
|
||||
// these — one note at a time, with the text as Markdown. That is a migration
|
||||
// nobody has to build a parser for.
|
||||
func (s *Server) registerNoteTools(m *mcp.Server) {
|
||||
m.Register(mcp.Tool{
|
||||
Name: "search_notes",
|
||||
Description: "Find notes by text. Matches title and body, and is generous about " +
|
||||
"Danish spelling — searching \"gron\" finds \"grøn\", and the other way round. " +
|
||||
"With no query it returns the most recent notes. This is the tool for " +
|
||||
"\"what did I write about the tax return\" or \"find my note on the kitchen\".",
|
||||
InputSchema: mcp.Schema(map[string]any{
|
||||
"text": mcp.Str("Words to look for in the title and body."),
|
||||
"limit": mcp.Int("How many to return. Defaults to 25, maximum 100."),
|
||||
}),
|
||||
}, s.mcpSearchNotes)
|
||||
|
||||
m.Register(mcp.Tool{
|
||||
Name: "create_note",
|
||||
Description: "Write a note. The body is Markdown and is stored as typed — it is " +
|
||||
"also exactly what an export produces, so nothing is converted and nothing " +
|
||||
"is lost. Two things in the text are read out of it and become links: " +
|
||||
"`#Projekt` files the note against that project, so it appears on the " +
|
||||
"project's page without anybody filing it there, and `[[Another note]]` " +
|
||||
"links to a note by its title. Use this to bring notes in from elsewhere — " +
|
||||
"Apple Notes, OneNote, a folder of Markdown — one call per note.",
|
||||
InputSchema: mcp.Schema(map[string]any{
|
||||
"body": mcp.Str("The note as Markdown. Its first line becomes the title, so start with one. #Projekt and [[note]] become links."),
|
||||
"project_id": mcp.Str("File it under a project by id. Usually unnecessary — a #tag in the body does the same and is what a person would type."),
|
||||
"pinned": mcp.Bool("Keep it at the top of the list."),
|
||||
}),
|
||||
}, s.mcpCreateNote)
|
||||
|
||||
m.Register(mcp.Tool{
|
||||
Name: "update_note",
|
||||
Description: "Change a note. Only the fields given are changed. Sending `body` " +
|
||||
"replaces the whole text, so read the note first if you mean to add to it — " +
|
||||
"and the links are read again from the new text, so a #tag removed is a link " +
|
||||
"removed.",
|
||||
InputSchema: mcp.Schema(map[string]any{
|
||||
"note_id": mcp.Str("Which note."),
|
||||
"body": mcp.Str("The whole new text, as Markdown. The title follows its first line."),
|
||||
"pinned": mcp.Bool("Pin it or unpin it."),
|
||||
}),
|
||||
}, s.mcpUpdateNote)
|
||||
}
|
||||
|
||||
func (s *Server) mcpSearchNotes(ctx context.Context, userID string, args json.RawMessage) (any, error) {
|
||||
var params struct {
|
||||
Text string `json:"text"`
|
||||
Limit int `json:"limit"`
|
||||
}
|
||||
_ = json.Unmarshal(args, ¶ms)
|
||||
|
||||
limit := params.Limit
|
||||
if limit <= 0 || limit > 100 {
|
||||
limit = 25
|
||||
}
|
||||
|
||||
var notes []store.Note
|
||||
var err error
|
||||
if strings.TrimSpace(params.Text) == "" {
|
||||
notes, err = s.db.LooseNotes(ctx, userID)
|
||||
} else {
|
||||
notes, err = s.db.SearchNotes(ctx, params.Text, limit)
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// The same filter the HTTP surface applies, for the same reason: a search must
|
||||
// not become a way to read what you cannot open.
|
||||
out := make([]map[string]any, 0, len(notes))
|
||||
for _, n := range notes {
|
||||
if !s.mcpMayRead(ctx, &n, userID) {
|
||||
continue
|
||||
}
|
||||
out = append(out, map[string]any{
|
||||
"id": n.ID, "title": n.Title, "body": n.Body,
|
||||
"project_id": n.ProjectID, "updated_at": n.UpdatedAt,
|
||||
})
|
||||
if len(out) >= limit {
|
||||
break
|
||||
}
|
||||
}
|
||||
return map[string]any{"notes": out, "count": len(out)}, nil
|
||||
}
|
||||
|
||||
func (s *Server) mcpMayRead(ctx context.Context, n *store.Note, userID string) bool {
|
||||
if n.ProjectID == "" {
|
||||
return n.CreatedBy == userID
|
||||
}
|
||||
_, err := store.RequireProjectRole(ctx, s.db, n.ProjectID, userID, store.RoleViewer)
|
||||
return err == nil
|
||||
}
|
||||
|
||||
func (s *Server) mcpCreateNote(ctx context.Context, userID string, args json.RawMessage) (any, error) {
|
||||
var params struct {
|
||||
Body string `json:"body"`
|
||||
ProjectID string `json:"project_id"`
|
||||
Pinned bool `json:"pinned"`
|
||||
}
|
||||
_ = json.Unmarshal(args, ¶ms)
|
||||
|
||||
n := &store.Note{
|
||||
CreatedBy: userID,
|
||||
Body: params.Body,
|
||||
Pinned: params.Pinned,
|
||||
}
|
||||
if params.ProjectID != "" {
|
||||
if _, err := store.RequireProjectRole(ctx, s.db, params.ProjectID, userID, store.RoleEditor); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
n.ProjectID = params.ProjectID
|
||||
}
|
||||
if err := s.db.SaveNote(ctx, n); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// The links are reported back rather than left silent. A note whose #tag was
|
||||
// misspelt looks identical to one whose tag landed, and in a chat nobody sees
|
||||
// the row that was written.
|
||||
return map[string]any{
|
||||
"id": n.ID, "title": n.Title, "links": n.Links,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (s *Server) mcpUpdateNote(ctx context.Context, userID string, args json.RawMessage) (any, error) {
|
||||
var params struct {
|
||||
NoteID string `json:"note_id"`
|
||||
Body *string `json:"body"`
|
||||
Pinned *bool `json:"pinned"`
|
||||
}
|
||||
_ = json.Unmarshal(args, ¶ms)
|
||||
|
||||
n, err := s.db.Note(ctx, params.NoteID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if n == nil || !s.mcpMayRead(ctx, n, userID) {
|
||||
return nil, fmt.Errorf("no such note")
|
||||
}
|
||||
if n.ProjectID != "" {
|
||||
if _, err := store.RequireProjectRole(ctx, s.db, n.ProjectID, userID, store.RoleEditor); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
if params.Body != nil {
|
||||
n.Body = *params.Body
|
||||
}
|
||||
if params.Pinned != nil {
|
||||
n.Pinned = *params.Pinned
|
||||
}
|
||||
if err := s.db.SaveNote(ctx, n); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return map[string]any{"id": n.ID, "title": n.Title, "links": n.Links}, nil
|
||||
}
|
||||
|
||||
@@ -0,0 +1,807 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"fmt"
|
||||
"io"
|
||||
"mime"
|
||||
"net/http"
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/go-chi/chi/v5"
|
||||
"github.com/kristianwind/verdande/internal/store"
|
||||
)
|
||||
|
||||
// Notes.
|
||||
//
|
||||
// Access follows the project, because that is where a note lives: a note in a
|
||||
// shared project is readable by everybody who can read the project and writable
|
||||
// by an editor. A note with no project is its author's alone — the same answer a
|
||||
// task in the inbox gets, arrived at the same way rather than by a rule of its own.
|
||||
|
||||
// mayReadNote reports whether this person may see the note, and mayWriteNote
|
||||
// whether they may change it. Split, because a viewer on a shared project can do
|
||||
// the first and not the second.
|
||||
func (s *Server) mayTouchNote(r *http.Request, n *store.Note, role store.Role) bool {
|
||||
user := userFrom(r.Context())
|
||||
if n.ProjectID == "" {
|
||||
return n.CreatedBy == user.ID
|
||||
}
|
||||
_, err := store.RequireProjectRole(r.Context(), s.db, n.ProjectID, user.ID, role)
|
||||
return err == nil
|
||||
}
|
||||
|
||||
func (s *Server) handleListNotes(w http.ResponseWriter, r *http.Request) {
|
||||
user := userFrom(r.Context())
|
||||
|
||||
// Searching and listing are the same endpoint, because they answer the same
|
||||
// question with different words. ?q= searches; ?project= narrows; neither gives
|
||||
// the loose ones.
|
||||
if q := strings.TrimSpace(r.URL.Query().Get("q")); q != "" {
|
||||
found, err := s.db.SearchNotes(r.Context(), q, 50)
|
||||
if err != nil {
|
||||
s.internal(w, r, "search notes", err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]any{"notes": briefly(s.readable(r, found))})
|
||||
return
|
||||
}
|
||||
|
||||
if projectID := r.URL.Query().Get("project"); projectID != "" {
|
||||
if _, err := store.RequireProjectRole(r.Context(), s.db, projectID, user.ID, store.RoleViewer); err != nil {
|
||||
s.storeError(w, r, "notes in project", err)
|
||||
return
|
||||
}
|
||||
found, err := s.db.NotesInProject(r.Context(), projectID)
|
||||
if err != nil {
|
||||
s.internal(w, r, "notes in project", err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]any{"notes": briefly(found)})
|
||||
return
|
||||
}
|
||||
|
||||
// The archive is the same list asked the other way round, so it is the same
|
||||
// endpoint with a flag rather than a second one that would have to be kept in
|
||||
// step with this.
|
||||
if r.URL.Query().Get("archived") != "" {
|
||||
found, err := s.db.ArchivedNotes(r.Context(), user.ID)
|
||||
if err != nil {
|
||||
s.internal(w, r, "list archived notes", err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]any{"notes": briefly(found)})
|
||||
return
|
||||
}
|
||||
|
||||
// Everything they can see, not only the loose ones. Filing a note in a project
|
||||
// is how it gets shared, and a list that dropped it at that moment made sharing
|
||||
// look like deleting.
|
||||
found, err := s.db.AllNotes(r.Context(), user.ID)
|
||||
if err != nil {
|
||||
s.internal(w, r, "list notes", err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]any{"notes": briefly(found)})
|
||||
}
|
||||
|
||||
// briefly replaces each note's body with the first of it.
|
||||
//
|
||||
// The list shows a title and about ninety characters. It was being sent whole
|
||||
// notes to do that: with twelve hundred of them the response was 845 KB, of which
|
||||
// ninety-eight per cent was body text nobody would read on that screen — fetched,
|
||||
// parsed and held in memory on every visit.
|
||||
//
|
||||
// Body is emptied rather than shortened, and that is the important half. A field
|
||||
// called `body` that holds a *piece* of the body is a trap: the editor would open
|
||||
// it, the debounce would save it, and the note would be cut down to its own
|
||||
// preview by nothing more than being looked at. Empty is a shape the client can
|
||||
// check, and it does — see the notes page, which fetches the whole note when it
|
||||
// opens one.
|
||||
//
|
||||
// Four hundred characters rather than ninety: the client strips Markdown before
|
||||
// it shows anything, and a heading marker and a bold word are characters that
|
||||
// vanish. Cheap insurance against a preview that ends up shorter than the line it
|
||||
// is meant to fill.
|
||||
const previewChars = 400
|
||||
|
||||
func briefly(notes []store.Note) []store.Note {
|
||||
out := make([]store.Note, 0, len(notes))
|
||||
for _, n := range notes {
|
||||
body := []rune(n.Body)
|
||||
if len(body) > previewChars {
|
||||
body = body[:previewChars]
|
||||
}
|
||||
n.Preview = string(body)
|
||||
n.Body = ""
|
||||
out = append(out, n)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// readable drops what this person may not see.
|
||||
//
|
||||
// Filtered after the search rather than inside it: FTS5 answers about text, and
|
||||
// teaching the index about roles would put an access rule in a place nobody would
|
||||
// think to look when it was wrong.
|
||||
func (s *Server) readable(r *http.Request, notes []store.Note) []store.Note {
|
||||
out := make([]store.Note, 0, len(notes))
|
||||
for _, n := range notes {
|
||||
if s.mayTouchNote(r, &n, store.RoleViewer) {
|
||||
out = append(out, n)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (s *Server) handleGetNote(w http.ResponseWriter, r *http.Request) {
|
||||
n, err := s.db.Note(r.Context(), chi.URLParam(r, "noteID"))
|
||||
if err != nil {
|
||||
s.internal(w, r, "get note", err)
|
||||
return
|
||||
}
|
||||
// The same answer for "no such note" and "not yours", so an id cannot be probed.
|
||||
if n == nil || !s.mayTouchNote(r, n, store.RoleViewer) {
|
||||
writeError(w, http.StatusNotFound, CodeNotFound, "no such note")
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, n)
|
||||
}
|
||||
|
||||
// No title. It is the note's first line and is derived on every save — a field
|
||||
// here would be a promise this server cannot keep.
|
||||
type noteBody struct {
|
||||
ProjectID *string `json:"project_id"`
|
||||
Body *string `json:"body"`
|
||||
Pinned *bool `json:"pinned"`
|
||||
}
|
||||
|
||||
func (s *Server) handleCreateNote(w http.ResponseWriter, r *http.Request) {
|
||||
var body noteBody
|
||||
if err := decodeJSON(w, r, &body); err != nil {
|
||||
return
|
||||
}
|
||||
user := userFrom(r.Context())
|
||||
|
||||
n := &store.Note{CreatedBy: user.ID}
|
||||
if body.ProjectID != nil && *body.ProjectID != "" {
|
||||
if _, err := store.RequireProjectRole(r.Context(), s.db, *body.ProjectID, user.ID, store.RoleEditor); err != nil {
|
||||
s.storeError(w, r, "create note", err)
|
||||
return
|
||||
}
|
||||
n.ProjectID = *body.ProjectID
|
||||
}
|
||||
if body.Body != nil {
|
||||
n.Body = *body.Body
|
||||
}
|
||||
if body.Pinned != nil {
|
||||
n.Pinned = *body.Pinned
|
||||
}
|
||||
|
||||
if err := s.db.SaveNote(r.Context(), n); err != nil {
|
||||
s.internal(w, r, "create note", err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusCreated, n)
|
||||
}
|
||||
|
||||
func (s *Server) handleUpdateNote(w http.ResponseWriter, r *http.Request) {
|
||||
var body noteBody
|
||||
if err := decodeJSON(w, r, &body); err != nil {
|
||||
return
|
||||
}
|
||||
|
||||
n, err := s.db.Note(r.Context(), chi.URLParam(r, "noteID"))
|
||||
if err != nil {
|
||||
s.internal(w, r, "update note", err)
|
||||
return
|
||||
}
|
||||
if n == nil || !s.mayTouchNote(r, n, store.RoleEditor) {
|
||||
writeError(w, http.StatusNotFound, CodeNotFound, "no such note")
|
||||
return
|
||||
}
|
||||
|
||||
if body.ProjectID != nil {
|
||||
// Moving it means being allowed on both sides: you cannot file somebody
|
||||
// else's note into your project, nor yours into theirs.
|
||||
if *body.ProjectID != "" {
|
||||
if _, err := store.RequireProjectRole(r.Context(), s.db, *body.ProjectID,
|
||||
userFrom(r.Context()).ID, store.RoleEditor); err != nil {
|
||||
s.storeError(w, r, "move note", err)
|
||||
return
|
||||
}
|
||||
}
|
||||
n.ProjectID = *body.ProjectID
|
||||
}
|
||||
if body.Body != nil {
|
||||
n.Body = *body.Body
|
||||
}
|
||||
if body.Pinned != nil {
|
||||
n.Pinned = *body.Pinned
|
||||
}
|
||||
|
||||
if err := s.db.SaveNote(r.Context(), n); err != nil {
|
||||
s.internal(w, r, "update note", err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, n)
|
||||
}
|
||||
|
||||
func (s *Server) handleDeleteNote(w http.ResponseWriter, r *http.Request) {
|
||||
n, err := s.db.Note(r.Context(), chi.URLParam(r, "noteID"))
|
||||
if err != nil {
|
||||
s.internal(w, r, "delete note", err)
|
||||
return
|
||||
}
|
||||
if n == nil || !s.mayTouchNote(r, n, store.RoleEditor) {
|
||||
writeError(w, http.StatusNotFound, CodeNotFound, "no such note")
|
||||
return
|
||||
}
|
||||
if err := s.db.DeleteNote(r.Context(), n.ID); err != nil {
|
||||
s.internal(w, r, "delete note", err)
|
||||
return
|
||||
}
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
}
|
||||
|
||||
// handleNotesLinking answers the backwards question for a task or a project:
|
||||
// what has somebody written about this.
|
||||
func (s *Server) handleNotesLinking(w http.ResponseWriter, r *http.Request) {
|
||||
kind := chi.URLParam(r, "kind")
|
||||
if kind != "task" && kind != "project" && kind != "note" {
|
||||
writeError(w, http.StatusBadRequest, CodeValidation, "kind must be task, project or note")
|
||||
return
|
||||
}
|
||||
found, err := s.db.NotesLinking(r.Context(), kind, chi.URLParam(r, "targetID"))
|
||||
if err != nil {
|
||||
s.internal(w, r, "notes linking", err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]any{"notes": s.readable(r, found)})
|
||||
}
|
||||
|
||||
// handleExportNotes writes every note this person can see as a zip of Markdown
|
||||
// files, one per note.
|
||||
//
|
||||
// A zip of plain files rather than one bundle in a format of our own: what comes
|
||||
// out is what was stored, and it opens in Obsidian, in a text editor, in anything.
|
||||
// That is the promise the whole design was arranged around — the note on disk is
|
||||
// already the file you would export — and this is where it is either kept or not.
|
||||
func (s *Server) handleExportNotes(w http.ResponseWriter, r *http.Request) {
|
||||
user := userFrom(r.Context())
|
||||
notes, err := s.db.AllNotes(r.Context(), user.ID)
|
||||
if err != nil {
|
||||
s.internal(w, r, "export notes", err)
|
||||
return
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "application/zip")
|
||||
w.Header().Set("Content-Disposition",
|
||||
`attachment; filename="verdande-noter-`+time.Now().Format("2006-01-02")+`.zip"`)
|
||||
|
||||
zw := zip.NewWriter(w)
|
||||
defer zw.Close()
|
||||
|
||||
// Names collide: two notes called "Møde" are two notes, and a zip with one entry
|
||||
// twice is a zip that loses one of them.
|
||||
used := map[string]int{}
|
||||
for _, n := range notes {
|
||||
name := noteFilename(n.Title)
|
||||
used[name]++
|
||||
if used[name] > 1 {
|
||||
name = fmt.Sprintf("%s (%d)", name, used[name])
|
||||
}
|
||||
|
||||
f, err := zw.CreateHeader(&zip.FileHeader{
|
||||
Name: name + ".md",
|
||||
Method: zip.Deflate,
|
||||
Modified: n.UpdatedAt,
|
||||
})
|
||||
if err != nil {
|
||||
// The response is already streaming, so there is no status left to send.
|
||||
// Stopping leaves a truncated zip, which a zip tool says is truncated —
|
||||
// better than a complete-looking file with a note missing from it.
|
||||
s.log.Error("export notes", "err", err, "note", n.ID)
|
||||
return
|
||||
}
|
||||
if _, err := io.WriteString(f, n.Body); err != nil {
|
||||
s.log.Error("export notes", "err", err, "note", n.ID)
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// noteFilename makes a title safe to be a filename on any of the three systems
|
||||
// people actually use, without turning it into something unrecognisable.
|
||||
func noteFilename(title string) string {
|
||||
name := strings.TrimSpace(title)
|
||||
if name == "" {
|
||||
name = "uden titel"
|
||||
}
|
||||
// Windows refuses these outright; a slash would make directories on the others.
|
||||
name = strings.Map(func(r rune) rune {
|
||||
if strings.ContainsRune(`/\:*?"<>|`, r) || r < 0x20 {
|
||||
return '-'
|
||||
}
|
||||
return r
|
||||
}, name)
|
||||
// Long titles are the first line of a note, which can be a whole sentence.
|
||||
if len([]rune(name)) > 80 {
|
||||
name = string([]rune(name)[:80])
|
||||
}
|
||||
return strings.TrimRight(name, " .")
|
||||
}
|
||||
|
||||
// maxNoteImportBytes bounds one upload. Generous for text — a decade of notes is
|
||||
// a few megabytes — and small enough that a wrong file is refused rather than read.
|
||||
const maxNoteImportBytes = 64 << 20
|
||||
|
||||
// What the archive may become once it is unpacked, and how many entries it may
|
||||
// hold. See handleImportNotes: the upload cap bounds what arrives, and says
|
||||
// nothing at all about what it expands to.
|
||||
const (
|
||||
maxNoteImportUnpacked = 2 << 30
|
||||
maxNoteImportFiles = 20_000
|
||||
)
|
||||
|
||||
// handleImportNotes reads a zip of Markdown files and makes a note of each.
|
||||
//
|
||||
// The mirror of the export, and deliberately the same shape: whatever comes out of
|
||||
// this program goes back into it. It is also the way in from anywhere else, since
|
||||
// a folder of Markdown is what Obsidian, Bear, iA Writer and a shell script all
|
||||
// produce — and what the Apple Notes exporter in tools/ writes.
|
||||
//
|
||||
// Filenames are ignored. The title is the first line of the body, everywhere else
|
||||
// in this program, and making it the filename here would be a second rule for the
|
||||
// same thing that disagrees the moment somebody renames a file.
|
||||
func (s *Server) handleImportNotes(w http.ResponseWriter, r *http.Request) {
|
||||
r.Body = http.MaxBytesReader(w, r.Body, maxNoteImportBytes)
|
||||
if err := r.ParseMultipartForm(4 << 20); err != nil {
|
||||
writeError(w, http.StatusRequestEntityTooLarge, CodePayloadTooLarge, "the file is too large")
|
||||
return
|
||||
}
|
||||
defer r.MultipartForm.RemoveAll()
|
||||
|
||||
file, header, err := r.FormFile("file")
|
||||
if err != nil {
|
||||
writeFieldErrors(w, map[string]string{"file": "required"})
|
||||
return
|
||||
}
|
||||
defer file.Close()
|
||||
|
||||
zr, err := zip.NewReader(file.(io.ReaderAt), header.Size)
|
||||
if err != nil {
|
||||
writeFieldErrors(w, map[string]string{"file": "not a zip archive"})
|
||||
return
|
||||
}
|
||||
|
||||
user := userFrom(r.Context())
|
||||
created, skipped, files := 0, 0, 0
|
||||
|
||||
// Two passes. Everything that is not Markdown is stored first, so that when a
|
||||
// note is read its pictures already have addresses to be rewritten to — a note
|
||||
// and its images arrive in whatever order the archive happens to hold them, and
|
||||
// one pass would leave half the links pointing at nothing.
|
||||
// The archive as a whole, not just the upload.
|
||||
//
|
||||
// The 64 MB cap is on what arrives; deflate reaches about 1032:1, so those 64
|
||||
// MB can become sixty-six gigabytes on the way out — written into the same
|
||||
// volume the database lives on. The upload limit says nothing about that, and
|
||||
// nothing else did either.
|
||||
//
|
||||
// Two ceilings: one per file, and one across the archive. A single enormous
|
||||
// photograph and ten thousand small ones are different attacks with the same
|
||||
// ending.
|
||||
if len(zr.File) > maxNoteImportFiles {
|
||||
writeError(w, http.StatusRequestEntityTooLarge, CodePayloadTooLarge,
|
||||
"that archive holds too many files")
|
||||
return
|
||||
}
|
||||
budget := int64(maxNoteImportUnpacked)
|
||||
|
||||
stored := map[string]string{}
|
||||
for _, f := range zr.File {
|
||||
name := f.Name
|
||||
if f.FileInfo().IsDir() || skipFile(name) || strings.EqualFold(path.Ext(name), ".md") {
|
||||
continue
|
||||
}
|
||||
// The header's own claim first, which costs nothing to check and rejects
|
||||
// the obvious case before a byte is written. It is only a claim, so the
|
||||
// reader below is limited as well.
|
||||
if f.UncompressedSize64 > maxUploadBytes {
|
||||
s.log.Warn("import: a file in the archive is too large", "file", name)
|
||||
skipped++
|
||||
continue
|
||||
}
|
||||
rc, err := f.Open()
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
hashed, size, err := s.storeUpload(io.LimitReader(rc, min(budget, maxUploadBytes)+1))
|
||||
rc.Close()
|
||||
if err != nil {
|
||||
s.log.Warn("import note file", "err", err, "file", name)
|
||||
continue
|
||||
}
|
||||
if size > maxUploadBytes || size > budget {
|
||||
// Over the line once it was actually read: remove what was written
|
||||
// rather than keeping a truncated file that would open as nothing.
|
||||
os.Remove(filepath.Join(s.cfg.FilesDir(), hashed))
|
||||
s.log.Warn("import: archive is larger unpacked than allowed", "file", name)
|
||||
skipped++
|
||||
continue
|
||||
}
|
||||
budget -= size
|
||||
stored[name] = hashed
|
||||
files++
|
||||
}
|
||||
|
||||
for _, f := range zr.File {
|
||||
name := f.Name
|
||||
if f.FileInfo().IsDir() || !strings.EqualFold(path.Ext(name), ".md") {
|
||||
continue
|
||||
}
|
||||
// Counted, not passed over in silence. A Markdown file the filter refuses is
|
||||
// a note that did not arrive, and the caller is told the same number either
|
||||
// way unless it is said here — which is how one note went missing without
|
||||
// anything looking wrong.
|
||||
if skipFile(name) {
|
||||
s.log.Warn("import skipped a note file", "file", name)
|
||||
skipped++
|
||||
continue
|
||||
}
|
||||
|
||||
rc, err := f.Open()
|
||||
if err != nil {
|
||||
skipped++
|
||||
continue
|
||||
}
|
||||
body, err := io.ReadAll(io.LimitReader(rc, 2<<20))
|
||||
rc.Close()
|
||||
if err != nil || strings.TrimSpace(string(body)) == "" {
|
||||
skipped++
|
||||
continue
|
||||
}
|
||||
|
||||
text, wrote, changed := readFrontMatter(string(body))
|
||||
|
||||
n := &store.Note{CreatedBy: user.ID, Body: text}
|
||||
if err := s.db.SaveNote(r.Context(), n); err != nil {
|
||||
s.log.Warn("import note", "err", err, "file", name)
|
||||
skipped++
|
||||
continue
|
||||
}
|
||||
|
||||
// After the save, not as part of it: SaveNote owns `updated_at` and is right
|
||||
// to, so the dates a note had before this program existed are written over
|
||||
// the top rather than passed through it.
|
||||
if !wrote.IsZero() || !changed.IsZero() {
|
||||
if err := s.db.SetNoteTimes(r.Context(), n.ID, wrote, changed); err != nil {
|
||||
s.log.Warn("import note dates", "err", err, "file", name)
|
||||
}
|
||||
}
|
||||
|
||||
// The pictures the note referred to, now that it has an id to hang them on.
|
||||
// The text is rewritten to the addresses they were given, so an image that
|
||||
// sat in the middle of a paragraph is still in the middle of it.
|
||||
if rewritten, attached := s.attachToNote(r, n, path.Dir(name), stored, zr); attached > 0 {
|
||||
n.Body = rewritten
|
||||
if err := s.db.SaveNote(r.Context(), n); err != nil {
|
||||
s.log.Warn("rewrite note links", "err", err, "note", n.ID)
|
||||
}
|
||||
}
|
||||
created++
|
||||
}
|
||||
|
||||
writeJSON(w, http.StatusOK, map[string]any{
|
||||
"created": created, "skipped": skipped, "files": files,
|
||||
})
|
||||
}
|
||||
|
||||
// skipFile is the things in an archive that are not content: directories' own
|
||||
// entries, macOS resource forks, and anything trying to climb out of the archive.
|
||||
// A zip made on a Mac carries a __MACOSX copy of every file, and importing those
|
||||
// would double a whole library with each copy full of binary rubbish.
|
||||
func skipFile(name string) bool {
|
||||
if strings.HasPrefix(name, "__MACOSX/") ||
|
||||
strings.HasPrefix(path.Base(name), "._") ||
|
||||
strings.HasPrefix(path.Base(name), ".") {
|
||||
return true
|
||||
}
|
||||
|
||||
// Traversal, read as path segments rather than as a substring.
|
||||
//
|
||||
// This was `strings.Contains(name, "..")`, which is the right idea aimed at the
|
||||
// wrong thing: it does not describe a path that climbs out of the archive, it
|
||||
// describes any name with two dots next to each other anywhere in it. A note
|
||||
// called "Så blev det endelig jul! Stay tuned... ☕️" was therefore dropped —
|
||||
// silently, and without even counting as skipped, so the number the import
|
||||
// reported still looked right. One note in twelve hundred, found only by
|
||||
// counting what came out.
|
||||
//
|
||||
// An ellipsis in a title is ordinary. A segment that *is* "..", or a path that
|
||||
// starts at the root, is not — and those are what a zip uses to write outside
|
||||
// the folder it was unpacked into.
|
||||
if strings.HasPrefix(name, "/") {
|
||||
return true
|
||||
}
|
||||
for _, part := range strings.Split(name, "/") {
|
||||
if part == ".." {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// attachToNote records the files a note's text points at and rewrites the text to
|
||||
// where they now live. Returns the new body and how many were attached.
|
||||
//
|
||||
// Only the files this note refers to. An archive is one folder for many notes, and
|
||||
// hanging every picture on every note would turn a library of a hundred into ten
|
||||
// thousand attachments.
|
||||
func (s *Server) attachToNote(
|
||||
r *http.Request, n *store.Note, dir string, stored map[string]string, zr *zip.Reader,
|
||||
) (string, int) {
|
||||
body := n.Body
|
||||
attached := 0
|
||||
|
||||
for _, f := range zr.File {
|
||||
hashed, ok := stored[f.Name]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
base := path.Base(f.Name)
|
||||
// The three ways a Markdown file names a neighbouring image: by its path
|
||||
// inside the archive, by the path relative to the note, and by its bare name.
|
||||
//
|
||||
// The archive path was missing, and it is the one an Apple Notes export
|
||||
// actually writes — `vedhaeftninger/751-2-IMG_0615.heic`, with the note at the
|
||||
// root beside the folder. Only the bare name matched, so only the bare name
|
||||
// was replaced, and every picture in every imported note ended up pointing at
|
||||
// `vedhaeftninger//api/v1/attachments/…`, which is nothing. The images were
|
||||
// stored and attached correctly; it was the link that was wrong.
|
||||
relative := strings.TrimPrefix(path.Join(dir, base), "./")
|
||||
refs := []string{f.Name, relative, base}
|
||||
if !strings.Contains(body, f.Name) &&
|
||||
!strings.Contains(body, relative) &&
|
||||
!strings.Contains(body, base) {
|
||||
continue
|
||||
}
|
||||
|
||||
a := &store.Attachment{
|
||||
NoteID: n.ID,
|
||||
Filename: base,
|
||||
MimeType: mimeOf(base),
|
||||
Size: int64(f.UncompressedSize64),
|
||||
Path: hashed,
|
||||
UploadedBy: n.CreatedBy,
|
||||
}
|
||||
if err := s.db.CreateAttachment(r.Context(), a); err != nil {
|
||||
s.log.Warn("attach to note", "err", err, "file", base)
|
||||
continue
|
||||
}
|
||||
|
||||
// Længste først. Erstattes det bare navn inde i en længere sti, bliver mappen
|
||||
// stående foran adressen, og linket peger på ingenting.
|
||||
url := "/api/v1/attachments/" + a.ID
|
||||
sort.Slice(refs, func(i, j int) bool { return len(refs[i]) > len(refs[j]) })
|
||||
for _, ref := range refs {
|
||||
if ref != "" {
|
||||
body = strings.ReplaceAll(body, ref, url)
|
||||
}
|
||||
}
|
||||
attached++
|
||||
}
|
||||
return body, attached
|
||||
}
|
||||
|
||||
// readFrontMatter takes the YAML block off the top of a Markdown file and reads
|
||||
// the two dates out of it, returning the rest of the text.
|
||||
//
|
||||
// Not a YAML parser, and deliberately not: the block this reads is the one this
|
||||
// program writes, plus whatever Obsidian and Bear write, and all of them use the
|
||||
// same handful of `key: value` lines. Pulling in a parser to read two dates would
|
||||
// be a dependency that can fail in ways nobody here would recognise.
|
||||
//
|
||||
// A file with no front matter comes back untouched — that is the normal case, and
|
||||
// it must never lose its first line to a rule about a block that is not there.
|
||||
func readFrontMatter(body string) (text string, created, modified time.Time) {
|
||||
if !strings.HasPrefix(body, "---\n") && !strings.HasPrefix(body, "---\r\n") {
|
||||
return body, time.Time{}, time.Time{}
|
||||
}
|
||||
rest := body[strings.Index(body, "\n")+1:]
|
||||
|
||||
end := strings.Index(rest, "\n---")
|
||||
if end < 0 {
|
||||
// An opening fence with no close is not front matter; it is a horizontal
|
||||
// rule somebody wrote at the top of their note.
|
||||
return body, time.Time{}, time.Time{}
|
||||
}
|
||||
block := rest[:end]
|
||||
after := rest[end+len("\n---"):]
|
||||
after = strings.TrimPrefix(after, "\r")
|
||||
after = strings.TrimPrefix(after, "\n")
|
||||
|
||||
for _, line := range strings.Split(block, "\n") {
|
||||
key, value, ok := strings.Cut(line, ":")
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
value = strings.TrimSpace(strings.Trim(strings.TrimSpace(value), `"'`))
|
||||
switch strings.ToLower(strings.TrimSpace(key)) {
|
||||
case "created", "created_at", "date":
|
||||
created = parseNoteTime(value)
|
||||
case "modified", "updated", "updated_at":
|
||||
modified = parseNoteTime(value)
|
||||
}
|
||||
}
|
||||
return strings.TrimLeft(after, "\n"), created, modified
|
||||
}
|
||||
|
||||
// parseNoteTime accepts the shapes these files actually carry. A date without a
|
||||
// time is that day at midnight, local — the note was written on that day, and
|
||||
// pretending to know the hour would be worse than not having one.
|
||||
func parseNoteTime(v string) time.Time {
|
||||
for _, layout := range []string{
|
||||
time.RFC3339,
|
||||
"2006-01-02T15:04:05",
|
||||
"2006-01-02 15:04:05",
|
||||
"2006-01-02 15:04",
|
||||
"2006-01-02",
|
||||
} {
|
||||
if t, err := time.ParseInLocation(layout, v, time.Local); err == nil {
|
||||
return t
|
||||
}
|
||||
}
|
||||
return time.Time{}
|
||||
}
|
||||
|
||||
// mimeOf guesses from the extension. The archive does not carry a type, and
|
||||
// sniffing the bytes would mean reading every file twice.
|
||||
func mimeOf(name string) string {
|
||||
if t := mime.TypeByExtension(path.Ext(name)); t != "" {
|
||||
return t
|
||||
}
|
||||
return "application/octet-stream"
|
||||
}
|
||||
|
||||
// handleUploadNoteAttachment takes a file into a note.
|
||||
//
|
||||
// The import could already do this, and nothing else could: a note brought in
|
||||
// from Apple Notes arrived with its pictures, and a note written here could not
|
||||
// be given one. Pasting a screenshot into a note is the most ordinary thing there
|
||||
// is, and it did nothing at all.
|
||||
//
|
||||
// Editor rights on the note, checked the same way every other write to it is. The
|
||||
// answer for "no such note" and "not yours" is the same, so an id cannot be
|
||||
// probed by watching which one comes back.
|
||||
func (s *Server) handleUploadNoteAttachment(w http.ResponseWriter, r *http.Request) {
|
||||
n, err := s.db.Note(r.Context(), chi.URLParam(r, "noteID"))
|
||||
if err != nil {
|
||||
s.internal(w, r, "get note", err)
|
||||
return
|
||||
}
|
||||
if n == nil || !s.mayTouchNote(r, n, store.RoleEditor) {
|
||||
writeError(w, http.StatusNotFound, CodeNotFound, "no such note")
|
||||
return
|
||||
}
|
||||
|
||||
r.Body = http.MaxBytesReader(w, r.Body, maxUploadBytes+(1<<20))
|
||||
if err := r.ParseMultipartForm(8 << 20); err != nil {
|
||||
writeError(w, http.StatusRequestEntityTooLarge, CodePayloadTooLarge,
|
||||
"that file is too large (25 MB maximum)")
|
||||
return
|
||||
}
|
||||
defer r.MultipartForm.RemoveAll()
|
||||
|
||||
file, header, err := r.FormFile("file")
|
||||
if err != nil {
|
||||
writeFieldErrors(w, map[string]string{"file": "required"})
|
||||
return
|
||||
}
|
||||
defer file.Close()
|
||||
|
||||
if header.Size > maxUploadBytes {
|
||||
writeError(w, http.StatusRequestEntityTooLarge, CodePayloadTooLarge,
|
||||
"that file is too large (25 MB maximum)")
|
||||
return
|
||||
}
|
||||
|
||||
stored, size, err := s.storeUpload(file)
|
||||
if err != nil {
|
||||
s.internal(w, r, "store upload", err)
|
||||
return
|
||||
}
|
||||
|
||||
user := userFrom(r.Context())
|
||||
a := &store.Attachment{
|
||||
NoteID: n.ID,
|
||||
Filename: safeFilename(header.Filename),
|
||||
MimeType: detectMime(header.Filename, header.Header.Get("Content-Type")),
|
||||
Size: size,
|
||||
Path: stored,
|
||||
UploadedBy: user.ID,
|
||||
}
|
||||
if err := s.db.CreateAttachment(r.Context(), a); err != nil {
|
||||
os.Remove(filepath.Join(s.cfg.FilesDir(), stored))
|
||||
s.internal(w, r, "record attachment", err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusCreated, toAttachmentJSON(*a))
|
||||
}
|
||||
|
||||
// --- putting notes away, one or many ---------------------------------------------
|
||||
|
||||
type noteBulkRequest struct {
|
||||
IDs []string `json:"ids"`
|
||||
Archived *bool `json:"archived"`
|
||||
Delete bool `json:"delete"`
|
||||
}
|
||||
|
||||
// maxNoteBulk bounds one call. Enough to select a screenful several times over,
|
||||
// and few enough that the loop below cannot become somebody's afternoon.
|
||||
const maxNoteBulk = 500
|
||||
|
||||
// handleNoteBulk archives, unarchives or deletes a set of notes in one request.
|
||||
//
|
||||
// One call rather than one per note, because the thing being asked is one thing:
|
||||
// "put these away". Fifty separate requests would be fifty round trips, fifty
|
||||
// chances for half of them to fail, and a list that redraws fifty times — and
|
||||
// after an import of twelve hundred notes, selecting fifty is the ordinary case,
|
||||
// not the extreme one.
|
||||
//
|
||||
// Each note is still checked on its own. A set is a convenience for the caller
|
||||
// and must never become a way to touch something they could not have touched one
|
||||
// at a time.
|
||||
func (s *Server) handleNoteBulk(w http.ResponseWriter, r *http.Request) {
|
||||
var req noteBulkRequest
|
||||
if err := decodeJSON(w, r, &req); err != nil {
|
||||
return
|
||||
}
|
||||
if len(req.IDs) == 0 {
|
||||
writeFieldErrors(w, map[string]string{"ids": "required"})
|
||||
return
|
||||
}
|
||||
if len(req.IDs) > maxNoteBulk {
|
||||
writeFieldErrors(w, map[string]string{"ids": "too many at once"})
|
||||
return
|
||||
}
|
||||
|
||||
done, skipped := 0, 0
|
||||
for _, id := range req.IDs {
|
||||
n, err := s.db.Note(r.Context(), id)
|
||||
if err != nil {
|
||||
s.internal(w, r, "get note", err)
|
||||
return
|
||||
}
|
||||
// Not an error, and not a 404 for the whole call: a set can contain a note
|
||||
// somebody else deleted a second ago, and the other forty-nine should still
|
||||
// be put away. Counted so the answer is honest about it.
|
||||
if n == nil || !s.mayTouchNote(r, n, store.RoleEditor) {
|
||||
skipped++
|
||||
continue
|
||||
}
|
||||
|
||||
switch {
|
||||
case req.Delete:
|
||||
err = s.db.DeleteNote(r.Context(), id)
|
||||
case req.Archived != nil:
|
||||
err = s.db.SetNoteArchived(r.Context(), id, *req.Archived)
|
||||
default:
|
||||
writeFieldErrors(w, map[string]string{"archived": "say archived or delete"})
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
s.internal(w, r, "note bulk", err)
|
||||
return
|
||||
}
|
||||
done++
|
||||
}
|
||||
|
||||
writeJSON(w, http.StatusOK, map[string]any{"done": done, "skipped": skipped})
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
package httpapi
|
||||
|
||||
import "testing"
|
||||
|
||||
// En titel med tre prikker i er ikke et forsøg på at skrive uden for mappen.
|
||||
//
|
||||
// `strings.Contains(name, "..")` var den rigtige idé rettet mod det forkerte:
|
||||
// den beskriver ikke en sti, der klatrer ud af arkivet, men ethvert navn med to
|
||||
// prikker ved siden af hinanden. Noten "Så blev det endelig jul! Stay tuned... ☕️"
|
||||
// blev derfor droppet lydløst — én ud af tolv hundrede, og kun fundet ved at
|
||||
// tælle efter, hvor mange der kom ud.
|
||||
func TestSkipFileTakesEllipsesButRefusesTraversal(t *testing.T) {
|
||||
keep := []string{
|
||||
"Så blev det endelig jul! Stay tuned... ☕️.md",
|
||||
"Møde 12.8..md",
|
||||
"vedhaeftninger/1-2-Pasted Graphic.png",
|
||||
"noter/Referat....md",
|
||||
}
|
||||
for _, name := range keep {
|
||||
if skipFile(name) {
|
||||
t.Errorf("skipFile(%q) = true, men det er et almindeligt filnavn", name)
|
||||
}
|
||||
}
|
||||
|
||||
drop := []string{
|
||||
"../uden for.md",
|
||||
"noter/../../etc/passwd",
|
||||
"/etc/passwd",
|
||||
"__MACOSX/._noget",
|
||||
".skjult.md",
|
||||
"noter/._AppleDouble.md",
|
||||
}
|
||||
for _, name := range drop {
|
||||
if !skipFile(name) {
|
||||
t.Errorf("skipFile(%q) = false, men den skal afvises", name)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,77 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"bytes"
|
||||
"io"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// En zip-bombe skal ikke kunne skrive gigabyte ned på den disk, databasen ligger
|
||||
// på.
|
||||
//
|
||||
// Uploaden er begrænset til 64 MB, og det siger ingenting om, hvad de 64 MB
|
||||
// bliver til: deflate når omkring 1032:1, så de kan pakkes ud til seksogtres
|
||||
// gigabyte. Prøven bygger en rigtig lille bombe — fire megabyte nuller, som
|
||||
// pakker til nogle få kilobyte — og ser efter, at loftet på det udpakkede holder.
|
||||
//
|
||||
// Skrevet imod tallene frem for imod endepunktet, fordi det, der skal holde, er
|
||||
// forholdet: en fil, hvis udpakkede størrelse overstiger loftet, må ikke læses
|
||||
// færdig.
|
||||
func TestAZipBombIsRefusedByItsUnpackedSize(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
zw := zip.NewWriter(&buf)
|
||||
w, err := zw.Create("bombe.bin")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Nuller pakker næsten helt væk. Fire megabyte er nok til at vise forholdet
|
||||
// uden at prøven selv bruger en gigabyte.
|
||||
if _, err := w.Write(make([]byte, 4<<20)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := zw.Close(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
packed := int64(buf.Len())
|
||||
if packed > 64<<10 {
|
||||
t.Fatalf("prøvens egen bombe pakkede kun til %d bytes; den viser ikke noget", packed)
|
||||
}
|
||||
|
||||
zr, err := zip.NewReader(bytes.NewReader(buf.Bytes()), packed)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
f := zr.File[0]
|
||||
if f.UncompressedSize64 != 4<<20 {
|
||||
t.Fatalf("udpakket størrelse = %d", f.UncompressedSize64)
|
||||
}
|
||||
|
||||
// Det, importen gør: læs højst det, budgettet tillader, plus én byte — så en
|
||||
// fil, der er præcis på grænsen, kan skelnes fra en, der er over.
|
||||
rc, err := f.Open()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer rc.Close()
|
||||
budget := int64(1 << 20)
|
||||
n, err := io.Copy(io.Discard, io.LimitReader(rc, budget+1))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if n <= budget {
|
||||
t.Fatalf("læste %d bytes af en fil på 4 MB med et budget på %d — loftet fanger den ikke", n, budget)
|
||||
}
|
||||
}
|
||||
|
||||
// Tallene skal stå i et forhold, der giver mening: en enkelt fil må aldrig kunne
|
||||
// bruge hele arkivets budget, og arkivet må ikke være mindre end én fil.
|
||||
func TestTheImportCeilingsAgreeWithEachOther(t *testing.T) {
|
||||
if maxNoteImportUnpacked <= maxUploadBytes {
|
||||
t.Error("arkivets budget er ikke større end én enkelt fil")
|
||||
}
|
||||
if maxNoteImportFiles < 2000 {
|
||||
t.Error("filloftet er under det, en rigtig Apple Noter-eksport indeholder")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,373 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A note in a project is the project's; a note in no project is its author's.
|
||||
// Both answers come from the project's own roles rather than from a rule notes
|
||||
// invented for themselves, which is the point of filing them there.
|
||||
func TestANoteIsNotReadableByJustAnybody(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
ts.bootstrap(t)
|
||||
|
||||
resp, body := ts.do(t, "POST", "/api/v1/notes", map[string]any{
|
||||
"body": "Noget privat om #Firma",
|
||||
})
|
||||
if resp.StatusCode != http.StatusCreated {
|
||||
t.Fatalf("create: %d %v", resp.StatusCode, body)
|
||||
}
|
||||
id, _ := body["id"].(string)
|
||||
if id == "" {
|
||||
t.Fatal("no id came back")
|
||||
}
|
||||
|
||||
// The tags were read out of the text on the way in, without being asked for.
|
||||
links, _ := body["links"].([]any)
|
||||
if len(links) != 1 {
|
||||
t.Fatalf("the note recorded %v", body["links"])
|
||||
}
|
||||
|
||||
// Somebody else gets the same answer as for a note that does not exist, so an
|
||||
// id cannot be probed for existence.
|
||||
other := ts.newUser(t, "[email protected]", "Anden")
|
||||
resp, _ = other.do(t, "GET", "/api/v1/notes/"+id, nil)
|
||||
if resp.StatusCode != http.StatusNotFound {
|
||||
t.Errorf("a stranger got %d for somebody else's note, want 404", resp.StatusCode)
|
||||
}
|
||||
}
|
||||
|
||||
// Searching must not become a way to read what you cannot open. The filter is
|
||||
// applied after the index, and this is what says so.
|
||||
func TestSearchDoesNotLeakNotesYouCannotRead(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
ts.bootstrap(t)
|
||||
|
||||
if resp, body := ts.do(t, "POST", "/api/v1/notes", map[string]any{
|
||||
"body": "Hemmeligt kodeord til hjemmesiden",
|
||||
}); resp.StatusCode != http.StatusCreated {
|
||||
t.Fatalf("create: %d %v", resp.StatusCode, body)
|
||||
}
|
||||
|
||||
_, mine := ts.do(t, "GET", "/api/v1/notes?q=hemmeligt", nil)
|
||||
if got := len(mine["notes"].([]any)); got != 1 {
|
||||
t.Fatalf("the author found %d of their own notes", got)
|
||||
}
|
||||
|
||||
other := ts.newUser(t, "[email protected]", "Anden")
|
||||
_, theirs := other.do(t, "GET", "/api/v1/notes?q=hemmeligt", nil)
|
||||
if got := len(theirs["notes"].([]any)); got != 0 {
|
||||
t.Errorf("a stranger's search returned %d notes", got)
|
||||
}
|
||||
}
|
||||
|
||||
// The backwards question, which is what makes notes worth linking at all.
|
||||
func TestANoteIsFoundFromTheTaskItMentions(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
ts.bootstrap(t)
|
||||
|
||||
_, task := ts.do(t, "POST", "/api/v1/tasks/quick-add", map[string]any{
|
||||
"text": "ring til Anders",
|
||||
})
|
||||
taskID, _ := task["id"].(string)
|
||||
if taskID == "" {
|
||||
t.Fatalf("no task id: %v", task)
|
||||
}
|
||||
|
||||
if resp, _ := ts.do(t, "POST", "/api/v1/notes", map[string]any{
|
||||
"body": "Aftalt at han ringer først, se /opgave/" + taskID,
|
||||
}); resp.StatusCode != http.StatusCreated {
|
||||
t.Fatal("the note was not written")
|
||||
}
|
||||
|
||||
_, found := ts.do(t, "GET", "/api/v1/notes/linking/task/"+taskID, nil)
|
||||
notes, _ := found["notes"].([]any)
|
||||
if len(notes) != 1 {
|
||||
t.Fatalf("%d notes point at the task, want 1", len(notes))
|
||||
}
|
||||
}
|
||||
|
||||
// Editing the text moves the links with it. A link that outlives its mention
|
||||
// claims a connection that is not there.
|
||||
func TestChangingANoteChangesWhatItPointsAt(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
ts.bootstrap(t)
|
||||
|
||||
_, note := ts.do(t, "POST", "/api/v1/notes", map[string]any{"body": "Om #Firma"})
|
||||
id, _ := note["id"].(string)
|
||||
|
||||
_, before := ts.do(t, "GET", "/api/v1/notes/linking/project/Firma", nil)
|
||||
if len(before["notes"].([]any)) != 1 {
|
||||
t.Fatal("the first save recorded nothing")
|
||||
}
|
||||
|
||||
if resp, _ := ts.do(t, "PATCH", "/api/v1/notes/"+id, map[string]any{
|
||||
"body": "Om #Regnskab i stedet",
|
||||
}); resp.StatusCode != http.StatusOK {
|
||||
t.Fatal("the note was not changed")
|
||||
}
|
||||
|
||||
_, after := ts.do(t, "GET", "/api/v1/notes/linking/project/Firma", nil)
|
||||
if got := len(after["notes"].([]any)); got != 0 {
|
||||
t.Errorf("%d notes still point at Firma", got)
|
||||
}
|
||||
}
|
||||
|
||||
// The export is the promise the whole design was arranged around: the note on
|
||||
// disk is already the file you would export, so nothing is converted on the way
|
||||
// out and nothing can be lost in the conversion.
|
||||
func TestNotesExportAsMarkdownFiles(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
ts.bootstrap(t)
|
||||
|
||||
bodies := []string{
|
||||
"# Møde med Anders\n\nHan vil gerne have **kaffe** hver uge.",
|
||||
"Slash/i/titlen\n\nog noget tekst",
|
||||
"# Møde med Anders\n\nen anden note med samme titel",
|
||||
}
|
||||
for _, body := range bodies {
|
||||
if resp, _ := ts.do(t, "POST", "/api/v1/notes", map[string]any{"body": body}); resp.StatusCode != http.StatusCreated {
|
||||
t.Fatalf("create: %d", resp.StatusCode)
|
||||
}
|
||||
}
|
||||
|
||||
req, _ := http.NewRequest("GET", ts.URL+"/api/v1/export/notes.zip", nil)
|
||||
req.Header.Set("Sec-Fetch-Site", "same-origin")
|
||||
for _, c := range ts.client.Jar.Cookies(mustParse(t, ts.URL)) {
|
||||
req.AddCookie(c)
|
||||
}
|
||||
resp, err := ts.client.Do(req)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
t.Fatalf("export: status %d", resp.StatusCode)
|
||||
}
|
||||
if ct := resp.Header.Get("Content-Type"); ct != "application/zip" {
|
||||
t.Errorf("content type is %q", ct)
|
||||
}
|
||||
|
||||
raw, err := io.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
zr, err := zip.NewReader(bytes.NewReader(raw), int64(len(raw)))
|
||||
if err != nil {
|
||||
t.Fatalf("the archive does not open: %v", err)
|
||||
}
|
||||
if len(zr.File) != 3 {
|
||||
t.Fatalf("the archive holds %d files, want 3", len(zr.File))
|
||||
}
|
||||
|
||||
byName := map[string]string{}
|
||||
for _, f := range zr.File {
|
||||
r, err := f.Open()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
text, _ := io.ReadAll(r)
|
||||
r.Close()
|
||||
byName[f.Name] = string(text)
|
||||
}
|
||||
|
||||
// Exactly what was stored, byte for byte. A converted export is an export that
|
||||
// can be wrong.
|
||||
if got := byName["Møde med Anders.md"]; got != bodies[0] {
|
||||
t.Errorf("the note came out as %q", got)
|
||||
}
|
||||
// A slash in a title would make a directory, and on Windows it is refused
|
||||
// outright.
|
||||
if _, ok := byName["Slash-i-titlen.md"]; !ok {
|
||||
t.Errorf("the awkward title became %v", keysOf(byName))
|
||||
}
|
||||
// Two notes with the same title are two notes; one entry twice is one lost.
|
||||
if _, ok := byName["Møde med Anders (2).md"]; !ok {
|
||||
t.Errorf("the second note of the same name is missing: %v", keysOf(byName))
|
||||
}
|
||||
}
|
||||
|
||||
func keysOf(m map[string]string) []string {
|
||||
out := make([]string, 0, len(m))
|
||||
for k := range m {
|
||||
out = append(out, k)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Out and back in again. If an export cannot be imported, it is not an export —
|
||||
// it is a file somebody hopes will be readable later.
|
||||
func TestNotesSurviveTheRoundTripThroughAZip(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
ts.bootstrap(t)
|
||||
|
||||
var archive bytes.Buffer
|
||||
zw := zip.NewWriter(&archive)
|
||||
files := map[string]string{
|
||||
"Møde med Anders.md": "# Møde med Anders\n\nHan vil have **kaffe** hver uge. #Firma",
|
||||
"noget andet.md": "Løse tanker\n\nog en linje til",
|
||||
// A zip made on a Mac carries these beside every file. Importing them would
|
||||
// double the whole library, with each copy full of binary rubbish.
|
||||
"__MACOSX/._Møde med Anders.md": "\x00\x05\x16\x07binært",
|
||||
"billede.png": "ikke en note",
|
||||
"tom.md": " \n ",
|
||||
}
|
||||
for name, body := range files {
|
||||
f, err := zw.Create(name)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := io.WriteString(f, body); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err := zw.Close(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
var form bytes.Buffer
|
||||
mw := multipart.NewWriter(&form)
|
||||
part, _ := mw.CreateFormFile("file", "noter.zip")
|
||||
part.Write(archive.Bytes())
|
||||
mw.Close()
|
||||
|
||||
req, _ := http.NewRequest("POST", ts.URL+"/api/v1/notes/import", &form)
|
||||
req.Header.Set("Content-Type", mw.FormDataContentType())
|
||||
req.Header.Set("Sec-Fetch-Site", "same-origin")
|
||||
for _, c := range ts.client.Jar.Cookies(mustParse(t, ts.URL)) {
|
||||
req.AddCookie(c)
|
||||
}
|
||||
resp, err := ts.client.Do(req)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
t.Fatalf("import: status %d", resp.StatusCode)
|
||||
}
|
||||
|
||||
var out struct{ Created, Skipped int }
|
||||
json.NewDecoder(resp.Body).Decode(&out)
|
||||
if out.Created != 2 {
|
||||
t.Fatalf("imported %d notes, want the 2 real ones", out.Created)
|
||||
}
|
||||
|
||||
// The title came from the first line, not from the filename — which is the same
|
||||
// rule as everywhere else, and the reason renaming a file cannot rename a note.
|
||||
_, listed := ts.do(t, "GET", "/api/v1/notes", nil)
|
||||
titles := map[string]bool{}
|
||||
for _, raw := range listed["notes"].([]any) {
|
||||
titles[raw.(map[string]any)["title"].(string)] = true
|
||||
}
|
||||
if !titles["Møde med Anders"] || !titles["Løse tanker"] {
|
||||
t.Errorf("the titles came out as %v", titles)
|
||||
}
|
||||
|
||||
// And the #tag was read on the way in, so an imported note lands on its project
|
||||
// exactly as a typed one does.
|
||||
_, linked := ts.do(t, "GET", "/api/v1/notes/linking/project/Firma", nil)
|
||||
if len(linked["notes"].([]any)) != 1 {
|
||||
t.Errorf("the imported note did not link to its project")
|
||||
}
|
||||
}
|
||||
|
||||
// A note from Apple Notes is full of photographs. One that arrives as words alone
|
||||
// is not that note — it is a summary of it.
|
||||
func TestAnImportedNoteKeepsItsPictures(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
ts.bootstrap(t)
|
||||
|
||||
var archive bytes.Buffer
|
||||
zw := zip.NewWriter(&archive)
|
||||
write := func(name, body string) {
|
||||
f, err := zw.Create(name)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := io.WriteString(f, body); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
write("noter/Kvittering.md", "# Kvittering\n\nHer er den:\n\n\n\nog resten af teksten")
|
||||
write("noter/vedhaeftninger/bon.png", "\x89PNG\r\n\x1a\nlad som om")
|
||||
// Another note in the same archive, which must not inherit the first one's
|
||||
// picture: an archive is one folder for many notes.
|
||||
write("noter/Uden billede.md", "Uden billede\n\nbare tekst")
|
||||
if err := zw.Close(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
var form bytes.Buffer
|
||||
mw := multipart.NewWriter(&form)
|
||||
part, _ := mw.CreateFormFile("file", "noter.zip")
|
||||
part.Write(archive.Bytes())
|
||||
mw.Close()
|
||||
|
||||
req, _ := http.NewRequest("POST", ts.URL+"/api/v1/notes/import", &form)
|
||||
req.Header.Set("Content-Type", mw.FormDataContentType())
|
||||
req.Header.Set("Sec-Fetch-Site", "same-origin")
|
||||
for _, c := range ts.client.Jar.Cookies(mustParse(t, ts.URL)) {
|
||||
req.AddCookie(c)
|
||||
}
|
||||
resp, err := ts.client.Do(req)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
var out struct{ Created, Skipped, Files int }
|
||||
json.NewDecoder(resp.Body).Decode(&out)
|
||||
if out.Created != 2 || out.Files != 1 {
|
||||
t.Fatalf("imported %d notes and %d files, want 2 and 1", out.Created, out.Files)
|
||||
}
|
||||
|
||||
// The list carries a preview, not the whole note — see briefly(). The body is
|
||||
// fetched by id, which is what the interface does when it opens one.
|
||||
_, listed := ts.do(t, "GET", "/api/v1/notes?q=kvittering", nil)
|
||||
notes := listed["notes"].([]any)
|
||||
if len(notes) != 1 {
|
||||
t.Fatalf("found %d notes", len(notes))
|
||||
}
|
||||
noteID := notes[0].(map[string]any)["id"].(string)
|
||||
if got := notes[0].(map[string]any)["body"]; got != nil && got != "" {
|
||||
t.Errorf("the list carried a body: %v — an editor handed that would save the truncation", got)
|
||||
}
|
||||
_, whole := ts.do(t, "GET", "/api/v1/notes/"+noteID, nil)
|
||||
body := whole["body"].(string)
|
||||
|
||||
// The link points at the file where it now lives, not at a path inside an
|
||||
// archive nobody kept.
|
||||
if strings.Contains(body, "vedhaeftninger/bon.png") {
|
||||
t.Errorf("the link still points into the archive: %q", body)
|
||||
}
|
||||
if !strings.Contains(body, "/api/v1/attachments/") {
|
||||
t.Errorf("the picture was not attached: %q", body)
|
||||
}
|
||||
// And it is still where it was written, in the middle of the note.
|
||||
if !strings.HasPrefix(body, "# Kvittering") || !strings.Contains(body, "og resten af teksten") {
|
||||
t.Errorf("the note was rearranged: %q", body)
|
||||
}
|
||||
|
||||
// The picture is really there and really readable.
|
||||
id := body[strings.Index(body, "/api/v1/attachments/")+len("/api/v1/attachments/"):]
|
||||
id = strings.SplitN(id, ")", 2)[0]
|
||||
got, _ := ts.do(t, "GET", "/api/v1/attachments/"+id, nil)
|
||||
if got.StatusCode != http.StatusOK {
|
||||
t.Errorf("the attachment answers %d", got.StatusCode)
|
||||
}
|
||||
|
||||
// The other note did not inherit it.
|
||||
_, other := ts.do(t, "GET", "/api/v1/notes?q=Uden", nil)
|
||||
for _, raw := range other["notes"].([]any) {
|
||||
if strings.Contains(raw.(map[string]any)["body"].(string), "attachments/") {
|
||||
t.Error("a note that mentioned no picture was given one")
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,12 +1,16 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/go-chi/chi/v5"
|
||||
|
||||
"github.com/kristianwind/verdande/internal/push"
|
||||
"github.com/kristianwind/verdande/internal/safedial"
|
||||
"github.com/kristianwind/verdande/internal/store"
|
||||
)
|
||||
|
||||
@@ -104,18 +108,44 @@ func (s *Server) PushToUser(userID, title, body, projectID string) {
|
||||
s.pushToUser(userID, title, body, projectID)
|
||||
}
|
||||
|
||||
func (s *Server) pushToUser(userID, title, body, projectID string) {
|
||||
// pushOutcome is what happened, per device.
|
||||
//
|
||||
// It exists because "nothing arrives" was unanswerable. A push that a service
|
||||
// refuses is the ordinary failure here — a stale subscription, a VAPID subject the
|
||||
// service will not take, a browser that revoked permission without telling the
|
||||
// server — and every one of them looked exactly like silence.
|
||||
type pushOutcome struct {
|
||||
Subscriptions int `json:"subscriptions"`
|
||||
Sent int `json:"sent"`
|
||||
Failed []pushFailure `json:"failed"`
|
||||
}
|
||||
|
||||
// pushFailure names the service and quotes what it said.
|
||||
//
|
||||
// The host and not the endpoint: the rest of that URL is the credential the push
|
||||
// service issued for this device, and it does not belong in an answer, a log line
|
||||
// or a screenshot.
|
||||
type pushFailure struct {
|
||||
Service string `json:"service"`
|
||||
Reason string `json:"reason"`
|
||||
}
|
||||
|
||||
func (s *Server) pushToUser(userID, title, body, projectID string) pushOutcome {
|
||||
ctx, cancel := contextWithBackgroundTimeout(30 * time.Second)
|
||||
defer cancel()
|
||||
|
||||
var out pushOutcome
|
||||
subs, err := s.db.ListPushSubscriptions(ctx, userID)
|
||||
if err != nil || len(subs) == 0 {
|
||||
return
|
||||
return out
|
||||
}
|
||||
out.Subscriptions = len(subs)
|
||||
|
||||
public, private, err := s.vapidKeys(ctx)
|
||||
if err != nil {
|
||||
s.log.Warn("vapid keys", "err", err)
|
||||
return
|
||||
out.Failed = append(out.Failed, pushFailure{Service: "verdande", Reason: err.Error()})
|
||||
return out
|
||||
}
|
||||
|
||||
payload := push.Payload{Title: title, Body: body, URL: "/projekt/" + projectID}
|
||||
@@ -126,13 +156,101 @@ func (s *Server) pushToUser(userID, title, body, projectID string) {
|
||||
Public: public, Private: private, Subject: "mailto:" + s.cfg.SMTP.From,
|
||||
})
|
||||
if err == nil {
|
||||
out.Sent++
|
||||
continue
|
||||
}
|
||||
s.log.Debug("web push failed", "err", err, "endpoint", sub.Endpoint)
|
||||
// Warn, not Debug.
|
||||
//
|
||||
// This was the one line that would have said why no notification ever
|
||||
// arrived, and at Debug it is not printed on an instance running at the
|
||||
// default level. A delivery that fails silently and a delivery that does
|
||||
// not happen look the same from the outside, and the difference is the
|
||||
// whole diagnosis.
|
||||
reason := pushReason(sub.Endpoint, err)
|
||||
s.log.Warn("web push refused", "reason", reason, "service", pushHost(sub.Endpoint), "user", userID)
|
||||
out.Failed = append(out.Failed, pushFailure{
|
||||
Service: pushHost(sub.Endpoint),
|
||||
Reason: reason,
|
||||
})
|
||||
if recErr := s.db.RecordPushFailure(ctx, sub.Endpoint, push.IsGone(err)); recErr != nil {
|
||||
s.log.Warn("record push failure", "err", recErr)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// pushReason is what went wrong, with the endpoint taken back out of it.
|
||||
//
|
||||
// Go wraps a transport failure as `*url.Error`, and its message is `Post
|
||||
// "https://fcm.googleapis.com/fcm/send/dQw4w9…": dial tcp: …` — the whole
|
||||
// endpoint, which is the credential the push service issued for that device. It
|
||||
// would have gone into the answer, the log line and any screenshot of either. A
|
||||
// test written to prove the host was reported instead is what found it.
|
||||
//
|
||||
// Unwrapped first, so the cause is what is quoted. Then the endpoint is scrubbed
|
||||
// anyway: the next error type to carry it will not come with a warning.
|
||||
func pushReason(endpoint string, err error) string {
|
||||
var urlErr *url.Error
|
||||
if errors.As(err, &urlErr) && urlErr.Err != nil {
|
||||
err = urlErr.Err
|
||||
}
|
||||
reason := err.Error()
|
||||
if endpoint != "" {
|
||||
reason = strings.ReplaceAll(reason, endpoint, pushHost(endpoint))
|
||||
}
|
||||
return reason
|
||||
}
|
||||
|
||||
// pushHost is the service an endpoint belongs to, and nothing else from it.
|
||||
func pushHost(endpoint string) string {
|
||||
u, err := url.Parse(endpoint)
|
||||
if err != nil || u.Host == "" {
|
||||
return "ukendt"
|
||||
}
|
||||
return u.Host
|
||||
}
|
||||
|
||||
// handlePushTest sends one notification to this account's devices and says what
|
||||
// happened to it.
|
||||
//
|
||||
// Reported as "der kommer ikke nogen". Everything about that was invisible: the
|
||||
// settings page said "on for this device" as soon as a subscription existed, and a
|
||||
// refusal from the push service went into a Debug line nobody sees. There was no
|
||||
// way to tell "the browser never subscribed" from "the service refused it" from
|
||||
// "nothing has come due yet" — three different problems with one appearance.
|
||||
//
|
||||
// It goes through the same path as a real notification rather than a shortcut, so
|
||||
// what it proves is the thing that actually has to work.
|
||||
//
|
||||
// The words come from the client, which is where the dictionaries are. A server
|
||||
// that carries its own copy of "This is a test" carries it in one language, and
|
||||
// this one deliberately keeps its prose in the interface — see
|
||||
// `internal/i18n`. The text is only ever shown back to the person who asked for
|
||||
// it, on their own devices.
|
||||
func (s *Server) handlePushTest(w http.ResponseWriter, r *http.Request) {
|
||||
var req struct {
|
||||
Title string `json:"title"`
|
||||
Body string `json:"body"`
|
||||
}
|
||||
if err := decodeJSON(w, r, &req); err != nil {
|
||||
return
|
||||
}
|
||||
title := strings.TrimSpace(req.Title)
|
||||
if title == "" {
|
||||
title = "verdande"
|
||||
}
|
||||
// Bounded: this is written into a notification on somebody's lock screen, and
|
||||
// a megabyte of title is not a notification.
|
||||
if len(title) > 200 {
|
||||
title = title[:200]
|
||||
}
|
||||
body := strings.TrimSpace(req.Body)
|
||||
if len(body) > 500 {
|
||||
body = body[:500]
|
||||
}
|
||||
|
||||
user := userFrom(r.Context())
|
||||
writeJSON(w, http.StatusOK, s.pushToUser(user.ID, title, body, ""))
|
||||
}
|
||||
|
||||
func (s *Server) vapidKeys(ctx contextLike) (string, string, error) {
|
||||
@@ -170,6 +288,23 @@ func (s *Server) handleSubscribePush(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
|
||||
// The endpoint is an address this server will POST to, on a schedule the
|
||||
// caller chooses — set a reminder a minute out and the push job fetches it,
|
||||
// again every minute. Unchecked, that is a way to make the instance reach
|
||||
// anything it can route to, on demand.
|
||||
//
|
||||
// https only: a push service is on the public internet and speaks TLS. Nothing
|
||||
// legitimate needs http here, and allowing it is the difference between a
|
||||
// blind request and a readable one.
|
||||
if !strings.HasPrefix(req.Endpoint, "https://") {
|
||||
writeFieldErrors(w, map[string]string{"endpoint": "must be an https address"})
|
||||
return
|
||||
}
|
||||
if reason := safedial.CheckURL(req.Endpoint); reason != "" {
|
||||
writeFieldErrors(w, map[string]string{"endpoint": reason})
|
||||
return
|
||||
}
|
||||
|
||||
err := s.db.SavePushSubscription(r.Context(), &store.PushSubscription{
|
||||
UserID: userFrom(r.Context()).ID,
|
||||
Endpoint: req.Endpoint,
|
||||
@@ -189,7 +324,10 @@ func (s *Server) handleUnsubscribePush(w http.ResponseWriter, r *http.Request) {
|
||||
if err := decodeJSON(w, r, &req); err != nil {
|
||||
return
|
||||
}
|
||||
if err := s.db.DeletePushSubscription(r.Context(), req.Endpoint); err != nil {
|
||||
// Scoped to the caller. It was not, so anybody who learned somebody else's
|
||||
// endpoint could unsubscribe their device — a small thing to do to somebody
|
||||
// and no reason at all to be able to.
|
||||
if err := s.db.DeletePushSubscription(r.Context(), userFrom(r.Context()).ID, req.Endpoint); err != nil {
|
||||
s.internal(w, r, "delete push subscription", err)
|
||||
return
|
||||
}
|
||||
|
||||
@@ -102,7 +102,7 @@ func TestTwoRunsOfOneMailboxDoNotOverlap(t *testing.T) {
|
||||
// Stands in for a run: takes the lock, notes how many are inside, and holds it
|
||||
// until told to let go.
|
||||
run := func(done chan<- struct{}) {
|
||||
unlock := srv.lockMailbox("box-1")
|
||||
unlock := srv.lockSync("box-1")
|
||||
mu.Lock()
|
||||
inside++
|
||||
if inside > most {
|
||||
@@ -136,10 +136,10 @@ func TestTwoRunsOfOneMailboxDoNotOverlap(t *testing.T) {
|
||||
|
||||
// And two different mailboxes must not wait for each other: one slow host is
|
||||
// not everybody's problem.
|
||||
a := srv.lockMailbox("box-a")
|
||||
a := srv.lockSync("box-a")
|
||||
done := make(chan struct{})
|
||||
go func() {
|
||||
srv.lockMailbox("box-b")()
|
||||
srv.lockSync("box-b")()
|
||||
close(done)
|
||||
}()
|
||||
select {
|
||||
|
||||
@@ -356,6 +356,34 @@ func (s *Server) handleCreateSection(w http.ResponseWriter, r *http.Request) {
|
||||
writeJSON(w, http.StatusCreated, sectionJSON{sec.ID, sec.ProjectID, sec.Name, sec.SortOrder})
|
||||
}
|
||||
|
||||
// handleReorderSections sets the order of the sections inside one project.
|
||||
//
|
||||
// Under /projects/{projectID}/ rather than beside the other section routes, which
|
||||
// hang off a section id: the thing being changed here is the project's list, not
|
||||
// any one section in it. That also puts it behind the same project permission
|
||||
// check as everything else in this subtree, instead of resolving a section to its
|
||||
// project one id at a time and hoping they all named the same one.
|
||||
func (s *Server) handleReorderSections(w http.ResponseWriter, r *http.Request) {
|
||||
var req reorderProjectsRequest
|
||||
if err := decodeJSON(w, r, &req); err != nil {
|
||||
return
|
||||
}
|
||||
// The same ceiling as the projects list, and a bound on the transaction rather
|
||||
// than an opinion about how many sections a project may have.
|
||||
if len(req.IDs) > 500 {
|
||||
writeFieldErrors(w, map[string]string{"ids": "at most 500 at a time"})
|
||||
return
|
||||
}
|
||||
|
||||
projectID := chi.URLParam(r, "projectID")
|
||||
if err := s.db.ReorderSections(r.Context(), projectID, req.IDs); err != nil {
|
||||
s.internal(w, r, "reorder sections", err)
|
||||
return
|
||||
}
|
||||
s.activity(r, projectID, "", "section.updated", nil)
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
}
|
||||
|
||||
// sectionAccess resolves a section to its project and checks the caller may edit
|
||||
// it. Sections have no permissions of their own — they belong to a project, and
|
||||
// that is where access is decided.
|
||||
|
||||
@@ -327,6 +327,71 @@ func TestAProjectCanHaveSeveralSections(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// Sections can be dragged into another order, and the order is the project's.
|
||||
//
|
||||
// The list is what a project's plan *is* — "Planlægning" before "I gang" is the
|
||||
// order the work happens in — and until this endpoint existed the only way to
|
||||
// change it was to delete a section and make it again, which moves its tasks to
|
||||
// the unsectioned area on the way past.
|
||||
func TestSectionsCanBeReordered(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
ts.bootstrap(t)
|
||||
|
||||
_, project := ts.do(t, "POST", "/api/v1/projects", map[string]any{"name": "Ombygning"})
|
||||
id := project["id"].(string)
|
||||
|
||||
ids := map[string]string{}
|
||||
names := []string{"Planlægning", "I gang", "Til gennemsyn"}
|
||||
for _, name := range names {
|
||||
_, body := ts.do(t, "POST", "/api/v1/projects/"+id+"/sections", map[string]any{"name": name})
|
||||
ids[name] = body["id"].(string)
|
||||
}
|
||||
|
||||
want := []string{"Til gennemsyn", "Planlægning", "I gang"}
|
||||
order := make([]string, len(want))
|
||||
for i, name := range want {
|
||||
order[i] = ids[name]
|
||||
}
|
||||
|
||||
resp, body := ts.do(t, "POST", "/api/v1/projects/"+id+"/sections/reorder",
|
||||
map[string]any{"ids": order})
|
||||
if resp.StatusCode != http.StatusNoContent {
|
||||
t.Fatalf("reorder: status %d, body %v", resp.StatusCode, body)
|
||||
}
|
||||
|
||||
_, listed := ts.do(t, "GET", "/api/v1/projects/"+id+"/sections", nil)
|
||||
got, _ := listed["sections"].([]any)
|
||||
if len(got) != len(want) {
|
||||
t.Fatalf("the project has %d sections, want %d", len(got), len(want))
|
||||
}
|
||||
for i, raw := range got {
|
||||
if name := raw.(map[string]any)["name"]; name != want[i] {
|
||||
t.Errorf("section %d is %v, want %q", i, name, want[i])
|
||||
}
|
||||
}
|
||||
|
||||
// A section belonging to somebody else's project is not moved by naming it
|
||||
// here. The ids in the body are not checked one by one — the WHERE clause is
|
||||
// what refuses them — so this is the assertion that it actually does.
|
||||
_, other := ts.do(t, "POST", "/api/v1/projects", map[string]any{"name": "Sommerhus"})
|
||||
otherID := other["id"].(string)
|
||||
_, foreign := ts.do(t, "POST", "/api/v1/projects/"+otherID+"/sections",
|
||||
map[string]any{"name": "Havearbejde"})
|
||||
foreignID := foreign["id"].(string)
|
||||
|
||||
resp, _ = ts.do(t, "POST", "/api/v1/projects/"+id+"/sections/reorder",
|
||||
map[string]any{"ids": []string{foreignID}})
|
||||
if resp.StatusCode != http.StatusNoContent {
|
||||
t.Fatalf("reorder with a foreign id: status %d", resp.StatusCode)
|
||||
}
|
||||
|
||||
_, stillThere := ts.do(t, "GET", "/api/v1/projects/"+otherID+"/sections", nil)
|
||||
elsewhere, _ := stillThere["sections"].([]any)
|
||||
if len(elsewhere) != 1 || elsewhere[0].(map[string]any)["name"] != "Havearbejde" {
|
||||
t.Errorf("the other project's section did not survive: %v", elsewhere)
|
||||
}
|
||||
}
|
||||
|
||||
// A project can be created straight into a group.
|
||||
//
|
||||
// `group_id` was declared on the request type, documented on it, and applied by
|
||||
|
||||
@@ -63,7 +63,15 @@ type Server struct {
|
||||
// the endpoints that check a secret are limited separately and more tightly
|
||||
// than the rest of the API.
|
||||
loginLimiter *limiter
|
||||
// Sat kun af prøver; se icsClient().
|
||||
icsFetch *http.Client
|
||||
mailLimiter *limiter
|
||||
resetLimiter *limiter
|
||||
// Second factors are counted against the *account*, not the address. An
|
||||
// address bucket is defeated by anybody with more than one address, which is
|
||||
// the ordinary case — and a six-digit code is only safe while guesses are
|
||||
// scarce, so the scarcity has to be counted against the thing being attacked.
|
||||
totpLimiter *limiter
|
||||
|
||||
// Computed once from the built index.html; see csp.go.
|
||||
csp string
|
||||
@@ -80,7 +88,16 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
|
||||
mail: mail.New(cfg.SMTP, cfg.BaseURL, log),
|
||||
hub: realtime.NewHub(log),
|
||||
loginLimiter: newLimiter(10, 15*time.Minute),
|
||||
// Generous on purpose: the caller is the mail server, one address delivering
|
||||
// everything, and a personal instance gets a handful of forwarded mails a
|
||||
// day. Sixty a minute never touches real delivery and still puts a ceiling
|
||||
// on the one public write path that has none.
|
||||
mailLimiter: newLimiter(60, time.Minute),
|
||||
resetLimiter: newLimiter(5, time.Hour),
|
||||
// Tighter than the password bucket, and for a whole hour. Somebody typing
|
||||
// their own code gets it right on the first or second try; ten in an hour is
|
||||
// generous for a person and hopeless for a search through a million.
|
||||
totpLimiter: newLimiter(10, time.Hour),
|
||||
}
|
||||
s.csp = contentSecurityPolicy(scriptHashes(web))
|
||||
if w, err := auth.NewWebAuthn(cfg.BaseURL, "verdande"); err == nil {
|
||||
@@ -113,7 +130,15 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
|
||||
|
||||
// Inbound mail. Delivered by the mail server rather than by a browser, and
|
||||
// authenticated by the token in the recipient address.
|
||||
r.Post("/inbound/mail", s.handleInboundMail)
|
||||
//
|
||||
// Rate limited for the same reason the beacon is: it is reachable by anybody
|
||||
// who can route to this instance. It is the weaker case of the two — the
|
||||
// beacon writes a row for a stranger, this one writes nothing without a
|
||||
// 256-bit address it has to already know — but an address is shared with mail
|
||||
// servers and printed on a settings page, and once it is known there was no
|
||||
// ceiling at all on what could be made with it.
|
||||
r.Method(http.MethodPost, "/inbound/mail",
|
||||
s.rateLimit(s.mailLimiter, http.HandlerFunc(s.handleInboundMail)))
|
||||
|
||||
// MCP with the token in the query string, for clients that cannot send a
|
||||
// header. Claude's custom-connector dialog takes a URL and nothing else, so
|
||||
@@ -133,6 +158,12 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
|
||||
r.Group(func(r chi.Router) {
|
||||
r.Use(s.requireAuth)
|
||||
r.Get("/oauth/gmail/callback", s.handleGmailCallback)
|
||||
// The calendar comes back on a URI of its own rather than sharing Gmail's.
|
||||
// One callback deciding from stored state which of the two flows had
|
||||
// returned would put the choice of *which token store to write* behind a
|
||||
// lookup — and when that goes wrong it goes wrong by overwriting a working
|
||||
// Gmail connection with calendar tokens.
|
||||
r.Get("/oauth/calendar/callback", s.handleCalendarCallback)
|
||||
})
|
||||
r.Route("/caldav", s.caldavRoutes)
|
||||
|
||||
@@ -143,6 +174,19 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
|
||||
writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
|
||||
})
|
||||
|
||||
// The beacon's receiving end, and the number it produces. Both without a
|
||||
// session, because the caller is a stranger's installation — that is the
|
||||
// whole feature. Rate limited like the login routes: it is the one endpoint
|
||||
// anybody on the internet can write to, and a row per request is a row per
|
||||
// request whether or not the request is honest.
|
||||
//
|
||||
// Refused with a 404 unless this instance has been told it is the collector,
|
||||
// so an ordinary installation is not a spam sink that answers 200 to
|
||||
// anything.
|
||||
r.Method(http.MethodPost, "/beacon",
|
||||
s.rateLimit(s.loginLimiter, http.HandlerFunc(s.handleBeaconPing)))
|
||||
r.Get("/beacon/count", s.handleBeaconCount)
|
||||
|
||||
// Reachable without being signed in — these are how you become signed in.
|
||||
r.Route("/auth", func(r chi.Router) {
|
||||
r.Get("/setup", s.handleSetupState)
|
||||
@@ -178,13 +222,10 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
|
||||
r.Get("/me", s.handleMe)
|
||||
r.Patch("/me", s.handleUpdateProfile)
|
||||
r.Put("/sidebar-sections", s.handleSetSidebarSections)
|
||||
r.Put("/nav-order", s.handleSetNavOrder)
|
||||
r.Post("/logout", s.handleLogout)
|
||||
r.Post("/password/change", s.handleChangePassword)
|
||||
r.Post("/totp/setup", s.handleTOTPSetup)
|
||||
r.Post("/totp/confirm", s.handleTOTPConfirm)
|
||||
r.Post("/totp/disable", s.handleTOTPDisable)
|
||||
r.Get("/recovery-codes", s.handleRecoveryCodesCount)
|
||||
r.Post("/recovery-codes", s.handleRecoveryCodesRegenerate)
|
||||
|
||||
// Behind requireSession, like the API tokens and for the same
|
||||
// reason: a leaked token must not be able to end the sessions of
|
||||
@@ -194,6 +235,23 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
|
||||
r.Get("/sessions", s.handleListSessions)
|
||||
r.Delete("/sessions/{sessionID}", s.handleDeleteSession)
|
||||
|
||||
// Turning the second factor on, and the codes that get you back
|
||||
// in. Sessions only, and setup asks for the password.
|
||||
//
|
||||
// These sat outside this group, which meant the door opened with
|
||||
// a token and closed only with a password: `disable` has always
|
||||
// asked for the password, `setup` asked for nothing. A stolen
|
||||
// token could therefore fetch the secret, load it into its own
|
||||
// authenticator, confirm it, and keep the only copy of the
|
||||
// recovery codes. The owner's next login would stop at a code
|
||||
// they do not have, and they could not turn it off without
|
||||
// logging in first. A theft becomes a tenancy — the same
|
||||
// sentence written next door about passkeys, and the same rule.
|
||||
r.Post("/totp/setup", s.handleTOTPSetup)
|
||||
r.Post("/totp/confirm", s.handleTOTPConfirm)
|
||||
r.Post("/totp/disable", s.handleTOTPDisable)
|
||||
r.Post("/recovery-codes", s.handleRecoveryCodesRegenerate)
|
||||
|
||||
// Registering a key, and the list of them. Sessions only, for
|
||||
// the same reason: a leaked token must not be able to add its
|
||||
// own way in — that would turn a theft into a tenancy.
|
||||
@@ -239,6 +297,9 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
|
||||
r.Get("/key", s.handlePushKey)
|
||||
r.Post("/subscribe", s.handleSubscribePush)
|
||||
r.Post("/unsubscribe", s.handleUnsubscribePush)
|
||||
// One notification, to this account's own devices, reporting what
|
||||
// each of them did with it. The answer to "der kommer ikke nogen".
|
||||
r.Post("/test", s.handlePushTest)
|
||||
})
|
||||
|
||||
r.Route("/attachments/{attachmentID}", func(r chi.Router) {
|
||||
@@ -276,6 +337,26 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
|
||||
// Mailboxes read over IMAP — iCloud, Fastmail, any ordinary host. Each
|
||||
// belongs to the person who connected it, so there is no admin gate
|
||||
// here and no shared registration: the credential is theirs.
|
||||
// Notes. Listing and searching are the same endpoint because they answer
|
||||
// the same question with different words; the backwards one — what has
|
||||
// been written about this task — hangs off the thing being asked about.
|
||||
r.Route("/notes", func(r chi.Router) {
|
||||
r.Get("/", s.handleListNotes)
|
||||
r.Post("/", s.handleCreateNote)
|
||||
r.Post("/import", s.handleImportNotes)
|
||||
r.Get("/linking/{kind}/{targetID}", s.handleNotesLinking)
|
||||
r.Get("/{noteID}", s.handleGetNote)
|
||||
r.Patch("/{noteID}", s.handleUpdateNote)
|
||||
r.Delete("/{noteID}", s.handleDeleteNote)
|
||||
// A picture pasted or dropped into a note. The import could bring
|
||||
// files in and nothing else could, which made the most ordinary act
|
||||
// in a notes app — paste a screenshot — do nothing at all.
|
||||
r.Post("/{noteID}/attachments", s.handleUploadNoteAttachment)
|
||||
// Mange på én gang. Efter en import af tolv hundrede noter er det at
|
||||
// vælge halvtreds det almindelige tilfælde, ikke det yderste.
|
||||
r.Post("/bulk", s.handleNoteBulk)
|
||||
})
|
||||
|
||||
r.Route("/mailboxes", func(r chi.Router) {
|
||||
r.Get("/", s.handleListMailboxes)
|
||||
r.Post("/", s.handleAddMailbox)
|
||||
@@ -302,12 +383,32 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
|
||||
})
|
||||
})
|
||||
|
||||
// A calendar read from somewhere else, laid under verdande's own dated
|
||||
// tasks. Read-only, so there is no write here to gate: the events are a
|
||||
// copy of somebody else's calendar, and the only thing a person decides
|
||||
// is which of them to look at.
|
||||
r.Route("/calendar", func(r chi.Router) {
|
||||
r.Get("/", s.handleGetCalendar)
|
||||
r.Put("/calendars", s.handleSetCalendars)
|
||||
r.Delete("/", s.handleDisconnectCalendar)
|
||||
r.Post("/authorize", s.handleCalendarAuthorize)
|
||||
r.Post("/sync", s.handleCalendarSyncNow)
|
||||
r.Get("/events", s.handleCalendarEvents)
|
||||
|
||||
// Abonnementer: en kalender, man kun har en adresse til. Flere pr.
|
||||
// person, hvor Google er én — det er adressen, der er identiteten,
|
||||
// hvor tokenet er det for OAuth.
|
||||
r.Post("/subscriptions", s.handleSubscribeCalendar)
|
||||
r.Delete("/subscriptions/{id}", s.handleUnsubscribeCalendar)
|
||||
})
|
||||
|
||||
r.Route("/import", func(r chi.Router) {
|
||||
r.Post("/todoist", s.handleImportTodoist)
|
||||
r.Post("/csv", s.handleImportCSV)
|
||||
})
|
||||
|
||||
r.Get("/export/account", s.handleExportAccount)
|
||||
r.Get("/export/notes.zip", s.handleExportNotes)
|
||||
r.Get("/export/projects/{projectID}.csv", s.handleExportProject)
|
||||
r.Get("/export/projects/{projectID}.ics", s.handleExportProjectICS)
|
||||
|
||||
@@ -350,6 +451,7 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
|
||||
r.Group(func(r chi.Router) {
|
||||
r.Use(s.requireProject(store.RoleEditor))
|
||||
r.Post("/sections", s.handleCreateSection)
|
||||
r.Post("/sections/reorder", s.handleReorderSections)
|
||||
})
|
||||
|
||||
r.Group(func(r chi.Router) {
|
||||
@@ -375,6 +477,11 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
|
||||
// `:latest` on the way, which is what makes a restart an update.
|
||||
r.Get("/panel", s.handlePanelStatus)
|
||||
r.Post("/panel/restart", s.handleRestartFromPanel)
|
||||
// Who this instance reports to, and whether it does. An
|
||||
// administrator's decision, not a person's: there is one
|
||||
// installation and it either phones home or it does not.
|
||||
r.Get("/beacon/settings", s.handleBeaconStatus)
|
||||
r.Put("/beacon/settings", s.handleSetBeacon)
|
||||
// A backup file is a complete copy of the database, so this whole
|
||||
// group is sessions-only as well as administrators-only: a leaked
|
||||
// token must not be able to download everybody's data.
|
||||
|
||||
@@ -6,6 +6,7 @@ import (
|
||||
"io"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
@@ -1777,3 +1778,42 @@ func TestQuickAddSectionIDIsCheckedAgainstTheProject(t *testing.T) {
|
||||
t.Errorf("content = %v", moved["content"])
|
||||
}
|
||||
}
|
||||
|
||||
// A word inside a word, on the task side.
|
||||
//
|
||||
// The notes search grew this first — a note holding `keepInventory` could not be
|
||||
// found by typing "keep inventory" — and a task is the same sentence with a
|
||||
// different table behind it. `SpeedAdmin` is one token, so "speed admin" asks for
|
||||
// two words that are nowhere.
|
||||
//
|
||||
// Having it in one of the two searches would be worse than having it in neither:
|
||||
// "it finds it in my notes but not in my tasks" is the next report.
|
||||
func TestATaskIsFoundByAWordInsideAWord(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
ts.bootstrap(t)
|
||||
|
||||
ts.do(t, "POST", "/api/v1/tasks", map[string]any{
|
||||
"content": "Opret kodeord til Kulturskolen (SpeedAdmin)",
|
||||
})
|
||||
ts.do(t, "POST", "/api/v1/tasks", map[string]any{"content": "Køb kaffe"})
|
||||
|
||||
for _, tc := range []struct {
|
||||
query string
|
||||
want int
|
||||
}{
|
||||
{"speed admin", 1},
|
||||
{"admin", 1},
|
||||
{"speedadmin", 1},
|
||||
{"kulturskolen", 1},
|
||||
{"vandhane", 0},
|
||||
} {
|
||||
resp, body := ts.do(t, "GET", "/api/v1/search?q="+url.QueryEscape(tc.query), nil)
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
t.Fatalf("%q: status %d", tc.query, resp.StatusCode)
|
||||
}
|
||||
tasks, _ := body["tasks"].([]any)
|
||||
if len(tasks) != tc.want {
|
||||
t.Errorf("%q found %d tasks, want %d", tc.query, len(tasks), tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -20,7 +20,24 @@ type totpSetupResponse struct {
|
||||
}
|
||||
|
||||
func (s *Server) handleTOTPSetup(w http.ResponseWriter, r *http.Request) {
|
||||
// The password, the same as disable has always asked for.
|
||||
//
|
||||
// Symmetry is the point. Turning the second factor on is at least as
|
||||
// consequential as turning it off — it decides which device gets to say yes
|
||||
// from now on — and a door that opens more easily than it closes is not a
|
||||
// door. Together with the move behind requireSession, this is what stops a
|
||||
// stolen credential from enrolling its own authenticator and keeping the only
|
||||
// copy of the recovery codes.
|
||||
var req totpDisableRequest
|
||||
if err := decodeJSON(w, r, &req); err != nil {
|
||||
return
|
||||
}
|
||||
user := userFrom(r.Context())
|
||||
if err := auth.VerifyPassword(user.PasswordHash, req.Password); err != nil {
|
||||
writeFieldErrors(w, map[string]string{"password": "not correct"})
|
||||
return
|
||||
}
|
||||
|
||||
if user.TOTPEnabled {
|
||||
writeError(w, http.StatusConflict, CodeConflict,
|
||||
"two-factor authentication is already switched on")
|
||||
|
||||
@@ -367,3 +367,180 @@ func unescape(s string) string {
|
||||
r := strings.NewReplacer(`\n`, "\n", `\N`, "\n", `\,`, ",", `\;`, ";", `\\`, `\`)
|
||||
return r.Replace(s)
|
||||
}
|
||||
|
||||
// --- reading a whole calendar ------------------------------------------------------
|
||||
|
||||
// Event is one VEVENT, reduced to what a grid needs — the same shape the Google
|
||||
// side already stores, so the two arrive in the database identically.
|
||||
//
|
||||
// Days at both ends, inclusive, because that is the question a grid asks: which
|
||||
// cells does this cover. StartsAt keeps the stamp as the file wrote it, offset and
|
||||
// all: it is what the chip shows, and reparsing it here would ask *this process*
|
||||
// what time zone it is in.
|
||||
type Event struct {
|
||||
UID string
|
||||
Summary string
|
||||
Location string
|
||||
URL string
|
||||
AllDay bool
|
||||
StartsAt string // RFC3339, empty for an all-day event
|
||||
EndsAt string
|
||||
StartDay string // YYYY-MM-DD
|
||||
EndDay string
|
||||
}
|
||||
|
||||
// Subscribed is what an .ics file somebody subscribed to says.
|
||||
//
|
||||
// Its own name rather than `Calendar`: that one is the *writing* side — what
|
||||
// verdande publishes as a feed — and the two are different documents that happen
|
||||
// to share a format. One name for both would be a promise that a round trip
|
||||
// through this package is lossless, which it is not and does not need to be.
|
||||
type Subscribed struct {
|
||||
Name string
|
||||
TimeZone string
|
||||
Events []Event
|
||||
}
|
||||
|
||||
// ParseCalendar reads an .ics file that somebody subscribed to.
|
||||
//
|
||||
// Forgiving in the same way ParseVTODO is, and for a sharper reason: this file was
|
||||
// written by somebody else's software and is fetched on a schedule with nobody
|
||||
// watching. One property this does not understand must not cost the other four
|
||||
// hundred events in the file — so anything unrecognised is skipped, and an event
|
||||
// without the two things a grid cannot do without (a uid and a start) is dropped
|
||||
// rather than stored half-made.
|
||||
//
|
||||
// # What it deliberately does not do
|
||||
//
|
||||
// **Recurrence is not expanded.** RRULE in a subscribed calendar means "and then
|
||||
// every Tuesday until 2031", and expanding it here would mean writing a year of
|
||||
// rows per rule and re-deciding what happens when the file changes. A recurring
|
||||
// event is stored as its first occurrence, which is honest — it is what the file
|
||||
// says on its own terms — and the day it matters, the answer is an expander shared
|
||||
// with `internal/recurrence`, not a second one written here.
|
||||
//
|
||||
// **Time zones are not resolved.** A DTSTART with TZID=Europe/Copenhagen is stored
|
||||
// with the offset the file carries or, when it carries none, as a local stamp. The
|
||||
// alternative is shipping a tz database and deciding what to do when the file
|
||||
// names a zone this build has never heard of. The day that matters, it is one
|
||||
// place.
|
||||
func ParseCalendar(body string) Subscribed {
|
||||
var cal Subscribed
|
||||
var current *Event
|
||||
depth := 0
|
||||
|
||||
for _, line := range unfold(body) {
|
||||
name, params, value := splitLine(line)
|
||||
upper := strings.ToUpper(name)
|
||||
|
||||
switch upper {
|
||||
case "BEGIN":
|
||||
if strings.EqualFold(value, "VEVENT") {
|
||||
current = &Event{}
|
||||
}
|
||||
depth++
|
||||
continue
|
||||
case "END":
|
||||
depth--
|
||||
if strings.EqualFold(value, "VEVENT") && current != nil {
|
||||
if current.UID != "" && current.StartDay != "" {
|
||||
cal.Events = append(cal.Events, *current)
|
||||
}
|
||||
current = nil
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
if current == nil {
|
||||
// Calendar-level properties. X-WR-CALNAME is what every publisher uses
|
||||
// to say what the calendar is called; there is no standard property for
|
||||
// it, which is why a non-standard one is read here.
|
||||
switch upper {
|
||||
case "X-WR-CALNAME":
|
||||
cal.Name = unescape(value)
|
||||
case "X-WR-TIMEZONE":
|
||||
cal.TimeZone = value
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
switch upper {
|
||||
case "UID":
|
||||
current.UID = value
|
||||
case "SUMMARY":
|
||||
current.Summary = unescape(value)
|
||||
case "LOCATION":
|
||||
current.Location = unescape(value)
|
||||
case "URL":
|
||||
current.URL = value
|
||||
case "DTSTART":
|
||||
current.AllDay = isDateOnly(params, value)
|
||||
current.StartDay, current.StartsAt = readStamp(params, value)
|
||||
case "DTEND":
|
||||
day, at := readStamp(params, value)
|
||||
current.EndsAt = at
|
||||
// DTEND is exclusive for an all-day event: a one-day event ends on the
|
||||
// next morning. Stored inclusive, or every all-day event would cover one
|
||||
// cell too many — and a public holiday would eat the day after it.
|
||||
if current.AllDay {
|
||||
day = shiftDay(day, -1)
|
||||
}
|
||||
current.EndDay = day
|
||||
}
|
||||
}
|
||||
|
||||
// A missing DTEND means the event ends when it starts. Filled after the loop
|
||||
// rather than inside it, because DTEND may arrive before DTSTART in a file
|
||||
// nobody promised would be in order.
|
||||
for i := range cal.Events {
|
||||
if cal.Events[i].EndDay == "" || cal.Events[i].EndDay < cal.Events[i].StartDay {
|
||||
cal.Events[i].EndDay = cal.Events[i].StartDay
|
||||
}
|
||||
}
|
||||
return cal
|
||||
}
|
||||
|
||||
// isDateOnly reports whether a stamp is a day rather than a moment. VALUE=DATE
|
||||
// says so outright; an eight-character value says it by being one.
|
||||
func isDateOnly(params, value string) bool {
|
||||
if strings.Contains(strings.ToUpper(params), "VALUE=DATE") {
|
||||
return true
|
||||
}
|
||||
return len(value) == 8 && !strings.ContainsAny(value, "T")
|
||||
}
|
||||
|
||||
// readStamp returns the day and, for a timed event, the moment as RFC3339.
|
||||
//
|
||||
// Three shapes exist in the wild: 20260821 (a day), 20260821T140000Z (UTC), and
|
||||
// 20260821T140000 with a TZID parameter (a local time in a named zone). The third
|
||||
// is kept as written, without a zone: resolving it needs a tz database, and a
|
||||
// stamp that says 14:00 is closer to the truth than one this process has moved.
|
||||
func readStamp(params, value string) (day, at string) {
|
||||
value = strings.TrimSpace(value)
|
||||
if len(value) < 8 {
|
||||
return "", ""
|
||||
}
|
||||
day = value[0:4] + "-" + value[4:6] + "-" + value[6:8]
|
||||
if len(value) < 15 || value[8] != 'T' {
|
||||
return day, ""
|
||||
}
|
||||
|
||||
clock := value[9:11] + ":" + value[11:13] + ":" + value[13:15]
|
||||
switch {
|
||||
case strings.HasSuffix(value, "Z"):
|
||||
return day, day + "T" + clock + "Z"
|
||||
default:
|
||||
// A local stamp, with or without a TZID. Written without an offset, which
|
||||
// is what the interface already treats as "the clock the file said".
|
||||
return day, day + "T" + clock
|
||||
}
|
||||
}
|
||||
|
||||
// shiftDay moves a YYYY-MM-DD by whole days.
|
||||
func shiftDay(day string, by int) string {
|
||||
t, err := time.Parse("2006-01-02", day)
|
||||
if err != nil {
|
||||
return day
|
||||
}
|
||||
return t.AddDate(0, 0, by).Format("2006-01-02")
|
||||
}
|
||||
|
||||
@@ -220,3 +220,101 @@ func TestFilename(t *testing.T) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// En abonnementskalender, som den ser ud fra nogen andens software.
|
||||
//
|
||||
// Alle fire former, der findes i naturen, i én fil: en heldagsbegivenhed, en med
|
||||
// et UTC-stempel, en med TZID og lokal tid, og en over flere dage. Plus en, der
|
||||
// mangler det, et gitter ikke kan undvære, og en foldet linje — begge dele
|
||||
// kommer, og de kommer uden varsel.
|
||||
func TestParseCalendarReadsTheFourShapesThatExist(t *testing.T) {
|
||||
body := "BEGIN:VCALENDAR\r\n" +
|
||||
"VERSION:2.0\r\n" +
|
||||
"X-WR-CALNAME:Helligdage\r\n" +
|
||||
"BEGIN:VEVENT\r\n" +
|
||||
"UID:[email protected]\r\n" +
|
||||
"DTSTART;VALUE=DATE:20260824\r\n" +
|
||||
"DTEND;VALUE=DATE:20260825\r\n" +
|
||||
"SUMMARY:Grundlovsdag\r\n" +
|
||||
"END:VEVENT\r\n" +
|
||||
"BEGIN:VEVENT\r\n" +
|
||||
"UID:[email protected]\r\n" +
|
||||
"DTSTART:20260825T120000Z\r\n" +
|
||||
"DTEND:20260825T133000Z\r\n" +
|
||||
`SUMMARY:Møde med en meget lang titel\, der er foldet over to lin` + "\r\n" + " jer\r\n" +
|
||||
"END:VEVENT\r\n" +
|
||||
"BEGIN:VEVENT\r\n" +
|
||||
"UID:[email protected]\r\n" +
|
||||
"DTSTART;TZID=Europe/Copenhagen:20260826T090000\r\n" +
|
||||
"DTEND;TZID=Europe/Copenhagen:20260826T100000\r\n" +
|
||||
"SUMMARY:Tandlæge\r\n" +
|
||||
"END:VEVENT\r\n" +
|
||||
"BEGIN:VEVENT\r\n" +
|
||||
"UID:[email protected]\r\n" +
|
||||
"DTSTART;VALUE=DATE:20260901\r\n" +
|
||||
"DTEND;VALUE=DATE:20260904\r\n" +
|
||||
"SUMMARY:Ferie\r\n" +
|
||||
"END:VEVENT\r\n" +
|
||||
"BEGIN:VEVENT\r\n" +
|
||||
"SUMMARY:Uden uid og uden start\r\n" +
|
||||
"END:VEVENT\r\n" +
|
||||
"END:VCALENDAR\r\n"
|
||||
|
||||
cal := ParseCalendar(body)
|
||||
|
||||
if cal.Name != "Helligdage" {
|
||||
t.Errorf("kalenderens navn = %q", cal.Name)
|
||||
}
|
||||
if len(cal.Events) != 4 {
|
||||
t.Fatalf("%d begivenheder, want 4 — den femte mangler både uid og start", len(cal.Events))
|
||||
}
|
||||
|
||||
// Heldag. DTEND er eksklusiv i filen: en endagsbegivenhed slutter næste morgen.
|
||||
// Gemmes inklusivt, ellers dækker hver eneste heldagsbegivenhed én celle for
|
||||
// meget — og en helligdag æder dagen efter sig.
|
||||
one := cal.Events[0]
|
||||
if !one.AllDay || one.StartDay != "2026-08-24" || one.EndDay != "2026-08-24" {
|
||||
t.Errorf("heldag: %+v", one)
|
||||
}
|
||||
if one.StartsAt != "" {
|
||||
t.Errorf("en heldagsbegivenhed fik et klokkeslæt: %q", one.StartsAt)
|
||||
}
|
||||
|
||||
// UTC, og den foldede linje samlet igen.
|
||||
two := cal.Events[1]
|
||||
if two.StartsAt != "2026-08-25T12:00:00Z" || two.EndsAt != "2026-08-25T13:30:00Z" {
|
||||
t.Errorf("utc: %+v", two)
|
||||
}
|
||||
if two.Summary != "Møde med en meget lang titel, der er foldet over to linjer" {
|
||||
t.Errorf("den foldede linje blev ikke samlet: %q", two.Summary)
|
||||
}
|
||||
|
||||
// Lokal tid med TZID: klokkeslættet står, som filen skrev det. At flytte det
|
||||
// her ville spørge *denne* proces, hvilken tidszone den er i.
|
||||
three := cal.Events[2]
|
||||
if three.StartsAt != "2026-08-26T09:00:00" {
|
||||
t.Errorf("lokal: %q", three.StartsAt)
|
||||
}
|
||||
|
||||
// Flere dage, begge ender inklusive.
|
||||
four := cal.Events[3]
|
||||
if four.StartDay != "2026-09-01" || four.EndDay != "2026-09-03" {
|
||||
t.Errorf("ferie: %s til %s", four.StartDay, four.EndDay)
|
||||
}
|
||||
}
|
||||
|
||||
// En fil, der ikke er en kalender, må ikke koste noget. Den hentes på et skema
|
||||
// uden nogen, der kigger med, og en fejl her ville være en sweep, der stopper.
|
||||
func TestParseCalendarSurvivesRubbish(t *testing.T) {
|
||||
for _, body := range []string{
|
||||
"",
|
||||
"ikke en kalender overhovedet",
|
||||
"BEGIN:VCALENDAR\r\nEND:VCALENDAR\r\n",
|
||||
"BEGIN:VEVENT\r\nDTSTART:x\r\nEND:VEVENT\r\n",
|
||||
"BEGIN:VEVENT\r\nUID:a\r\nDTSTART:2026\r\nEND:VEVENT\r\n",
|
||||
} {
|
||||
if got := ParseCalendar(body); len(got.Events) != 0 {
|
||||
t.Errorf("%q gav %d begivenheder", body, len(got.Events))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -310,3 +310,61 @@ func cut(text string) string {
|
||||
}
|
||||
return text
|
||||
}
|
||||
|
||||
// Unflag takes the flag off the messages that have become tasks.
|
||||
//
|
||||
// This is what stops the same mail arriving twice, and it replaces a marker that
|
||||
// could not be made to hold. The marker was a uid: remember the highest one seen,
|
||||
// start above it next time. That is correct only if uids are what the server says
|
||||
// they are, and it broke in every way such a scheme can — a fetch reply without a
|
||||
// uid, a UIDVALIDITY that changed under a rebuilt mailbox, two mailbox rows
|
||||
// pointing at the same folder and each keeping its own marker. Every one of those
|
||||
// produced the same symptom: the same mail became a task again, and again.
|
||||
//
|
||||
// Unflagging moves the record out of our database and into the mailbox, where the
|
||||
// question is asked. "Flagged" means "turn this into a task"; once it is a task,
|
||||
// the flag has done its job and is not true any more. Nothing has to be
|
||||
// remembered, so there is nothing to get out of step — a second sync, a second
|
||||
// mailbox row, or a database restored from a backup all find the same answer,
|
||||
// because the answer is in the mailbox.
|
||||
//
|
||||
// It is destructive in the small way that matters: it changes something in
|
||||
// somebody's mail. That is the trade, and it is the right way round — a flag is a
|
||||
// note to self that this needs doing, and it has been done.
|
||||
//
|
||||
// The folder is re-selected writable. Since() opens it read-only on purpose, and
|
||||
// a search should not need write access just because a later step might.
|
||||
func (c *Client) Unflag(uids []uint32) error {
|
||||
if len(uids) == 0 {
|
||||
return nil
|
||||
}
|
||||
if _, err := c.c.Select(c.folder, nil).Wait(); err != nil {
|
||||
return fmt.Errorf("open %s for writing: %w", c.folder, err)
|
||||
}
|
||||
|
||||
set := imap.UIDSetNum()
|
||||
for _, u := range uids {
|
||||
if u == 0 {
|
||||
// No uid means nothing to address. The message keeps its flag and will
|
||||
// be seen again — which is the failure this whole function exists to
|
||||
// prevent, so it is worth saying rather than skipping quietly.
|
||||
continue
|
||||
}
|
||||
set.AddNum(imap.UID(u))
|
||||
}
|
||||
if len(set) == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
// Silent: the server has nothing to tell us that we would act on, and the
|
||||
// untagged replies would only have to be collected and thrown away.
|
||||
cmd := c.c.Store(set, &imap.StoreFlags{
|
||||
Op: imap.StoreFlagsDel,
|
||||
Silent: true,
|
||||
Flags: []imap.Flag{imap.FlagFlagged},
|
||||
}, nil)
|
||||
if err := cmd.Close(); err != nil {
|
||||
return fmt.Errorf("unflag in %s: %w", c.folder, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -316,3 +316,81 @@ func TestNothingAtOrBelowTheMarkerComesBack(t *testing.T) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Den samme mail må ikke kunne blive en opgave to gange.
|
||||
//
|
||||
// Prøven kører hele vejen imod en rigtig server: den finder de flagede mails,
|
||||
// tager flaget af dem, og spørger så forfra som en ny synkronisering ville — med
|
||||
// markøren sat til nul, altså det værst tænkelige tilfælde, hvor alt, hvad vi
|
||||
// selv havde husket, er væk. Der skal ikke komme noget tilbage.
|
||||
//
|
||||
// Det er det, markøren ikke kunne love. Den holdt kun, hvis uids var, hvad
|
||||
// serveren sagde, og fejlede på et fetch-svar uden uid, på en UIDVALIDITY, der
|
||||
// skiftede, og på to postkasse-rækker, der pegede samme sted og hver især førte
|
||||
// deres eget regnskab. Alle tre gav det samme: mailen blev en opgave igen.
|
||||
func TestAnUnflaggedMailDoesNotComeBack(t *testing.T) {
|
||||
addr, pool := serveMailbox(t, []mail{
|
||||
{subject: "Faktura 4711", from: "[email protected]", body: "Vedhæftet.", flagged: true},
|
||||
{subject: "Nyhedsbrev", from: "[email protected]", body: "Læs mere", flagged: false},
|
||||
{subject: "Kontrakt", from: "[email protected]", body: "Til underskrift", flagged: true},
|
||||
})
|
||||
|
||||
client, err := Dial(Account{Host: addr, Username: "kw", Password: "hemmelig", RootCAs: pool})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer client.Close()
|
||||
|
||||
msgs, _, err := client.Since(0, 25)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(msgs) != 2 {
|
||||
t.Fatalf("fandt %d flagede mails, ventede 2", len(msgs))
|
||||
}
|
||||
|
||||
uids := make([]uint32, 0, len(msgs))
|
||||
for _, m := range msgs {
|
||||
uids = append(uids, m.UID)
|
||||
}
|
||||
if err := client.Unflag(uids); err != nil {
|
||||
t.Fatalf("Unflag: %v", err)
|
||||
}
|
||||
|
||||
// Forfra, og fra nul: selv en instans, der intet husker, må ikke finde dem igen.
|
||||
again, _, err := client.Since(0, 25)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(again) != 0 {
|
||||
t.Fatalf("%d mails kom igen efter at flaget var taget af — den første af dem er %q",
|
||||
len(again), again[0].Subject)
|
||||
}
|
||||
}
|
||||
|
||||
// Et tomt kald må ikke åbne mappen skrivbar for ingenting.
|
||||
func TestUnflagWithNothingToDoIsQuiet(t *testing.T) {
|
||||
addr, pool := serveMailbox(t, []mail{
|
||||
{subject: "Faktura", from: "[email protected]", body: "x", flagged: true},
|
||||
})
|
||||
client, err := Dial(Account{Host: addr, Username: "kw", Password: "hemmelig", RootCAs: pool})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer client.Close()
|
||||
|
||||
if err := client.Unflag(nil); err != nil {
|
||||
t.Errorf("Unflag(nil) = %v", err)
|
||||
}
|
||||
// Kun nuller: der er intet at adressere, og mailen skal beholde sit flag.
|
||||
if err := client.Unflag([]uint32{0, 0}); err != nil {
|
||||
t.Errorf("Unflag(nuller) = %v", err)
|
||||
}
|
||||
msgs, _, err := client.Since(0, 25)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(msgs) != 1 {
|
||||
t.Errorf("mailen mistede sit flag, uden at nogen havde adresseret den")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -41,10 +41,19 @@ type Runner struct {
|
||||
// reason as SyncGmail: it owns turning a message into a task.
|
||||
SyncMailbox func(ctx context.Context, user *store.User, m *store.Mailbox) (int, error)
|
||||
|
||||
// SyncCalendars refreshes one person's connected calendars. Supplied the same
|
||||
// way and for the same reason: the HTTP layer owns the token refresh.
|
||||
SyncCalendars func(ctx context.Context, user *store.User) (int, error)
|
||||
|
||||
// Push delivers a notification to somebody's devices. Supplied by the HTTP
|
||||
// layer, which owns the VAPID keys and the subscription list.
|
||||
Push func(userID, title, body, projectID string)
|
||||
|
||||
// SendBeacon reports this installation to the collector, if it is switched on
|
||||
// and a day has passed. Supplied by the HTTP layer for the same reason as the
|
||||
// syncs: that is where the instance settings and the version live.
|
||||
SendBeacon func(ctx context.Context) error
|
||||
|
||||
wg sync.WaitGroup
|
||||
}
|
||||
|
||||
@@ -69,6 +78,17 @@ func (r *Runner) Start(ctx context.Context) {
|
||||
// account for a to-do app. Ten minutes is fast enough for something somebody
|
||||
// starred and slow enough not to look like abuse.
|
||||
r.every(ctx, "mailboxes", 10*time.Minute, r.syncMailboxes)
|
||||
// Fifteen minutes rather than ten. A calendar is not a mailbox: nothing here
|
||||
// becomes a task, so being a quarter of an hour behind on a meeting next
|
||||
// Tuesday costs nothing — and the "fetch now" button covers the one case where
|
||||
// it matters, which is somebody who has just moved something and is looking at
|
||||
// the grid to check. A person who has just connected does not wait for this
|
||||
// either; connecting fetches the list, and the first sync is one click away.
|
||||
r.every(ctx, "calendars", 15*time.Minute, r.syncCalendars)
|
||||
// Checked hourly, sent at most daily — the same shape as the backup, and for
|
||||
// the same reason: a container asleep at the scheduled minute would otherwise
|
||||
// skip the day. SendBeacon decides whether a day has passed.
|
||||
r.every(ctx, "beacon", time.Hour, r.sendBeacon)
|
||||
}
|
||||
|
||||
func (r *Runner) Wait() { r.wg.Wait() }
|
||||
@@ -103,6 +123,17 @@ func (r *Runner) every(ctx context.Context, name string, interval time.Duration,
|
||||
}()
|
||||
}
|
||||
|
||||
// sendBeacon reports that this installation exists, at most once a day.
|
||||
//
|
||||
// Every decision about what is sent, and why this is on by default rather than
|
||||
// compulsory, lives in internal/beacon. This is only the clock.
|
||||
func (r *Runner) sendBeacon(ctx context.Context) error {
|
||||
if r.SendBeacon == nil {
|
||||
return nil
|
||||
}
|
||||
return r.SendBeacon(ctx)
|
||||
}
|
||||
|
||||
// --- reminders ------------------------------------------------------------------
|
||||
|
||||
// sendDueReminders delivers everything that has come due since the last pass.
|
||||
@@ -322,6 +353,44 @@ func (r *Runner) syncMailboxes(ctx context.Context) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// --- calendars ------------------------------------------------------------------------
|
||||
|
||||
// syncCalendars refreshes everybody's connected calendars.
|
||||
//
|
||||
// Its own sweep rather than a branch inside the mailbox one. They share nothing
|
||||
// past the word "poll": that loop asks each mailbox how far it has read and makes
|
||||
// tasks, this one replaces a window of somebody else's events. Folding them
|
||||
// together would mean one interval for two questions that do not want the same
|
||||
// answer, and a Google that is down would count against a mailbox that is not.
|
||||
//
|
||||
// Nobody is waiting, so no budget of its own — the request handler has one because
|
||||
// a person is watching a spinner. One account that has gone quiet costs this run
|
||||
// its own timeout and nothing else: the loop moves on per person.
|
||||
func (r *Runner) syncCalendars(ctx context.Context) error {
|
||||
if r.SyncCalendars == nil {
|
||||
return nil
|
||||
}
|
||||
userIDs, err := r.db.UsersWithCalendars(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, id := range userIDs {
|
||||
user, err := r.db.UserByID(ctx, id)
|
||||
if err != nil || user == nil {
|
||||
continue
|
||||
}
|
||||
written, err := r.SyncCalendars(ctx, user)
|
||||
if err != nil {
|
||||
r.log.Warn("calendar sync", "err", err, "user", id)
|
||||
continue
|
||||
}
|
||||
if written > 0 {
|
||||
r.log.Info("calendar refreshed", "user", id, "events", written)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// --- housekeeping -----------------------------------------------------------------
|
||||
|
||||
func (r *Runner) emptyTrash(ctx context.Context) error {
|
||||
|
||||
@@ -32,6 +32,8 @@ import (
|
||||
"net/url"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/kristianwind/verdande/internal/safedial"
|
||||
)
|
||||
|
||||
type Subscription struct {
|
||||
@@ -101,7 +103,19 @@ func Send(ctx context.Context, sub Subscription, payload Payload, vapid VAPID) e
|
||||
req.Header.Set("Urgency", "normal")
|
||||
req.Header.Set("Authorization", "vapid t="+token+", k="+vapid.Public)
|
||||
|
||||
resp, err := (&http.Client{Timeout: 15 * time.Second}).Do(req)
|
||||
// safedial, not a plain client. The endpoint comes from the browser's
|
||||
// subscription, which is to say from whoever is signed in, and this server
|
||||
// POSTs to it on a schedule they choose — set a reminder a minute out and the
|
||||
// job goes again every minute.
|
||||
//
|
||||
// The subscribe handler already refuses a private address, and that check is
|
||||
// not enough on its own: it runs on the URL as written, and a name it approves
|
||||
// can answer 127.0.0.1 the next time it is looked up. safedial checks the
|
||||
// *resolved* address at connect time, which is the only moment the address
|
||||
// that will actually be used is known — and it checks every redirect hop the
|
||||
// same way. The parse-time check stays, because it says no while somebody is
|
||||
// looking at the screen.
|
||||
resp, err := safedial.Client(15 * time.Second).Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -0,0 +1,66 @@
|
||||
package push
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/ecdh"
|
||||
"crypto/rand"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A subscription must not be able to point the server at its own network.
|
||||
//
|
||||
// The endpoint is whatever the browser handed over, which is to say whatever the
|
||||
// person signed in handed over, and this server POSTs to it on a schedule they
|
||||
// choose. The subscribe handler refuses a private address when it is written
|
||||
// down — and that check runs on the URL as written, so a name it approves can
|
||||
// answer 127.0.0.1 the next time it is looked up. This is the other half: the
|
||||
// address that will actually be dialled, checked at the moment of dialling.
|
||||
func TestSendRefusesToReachInside(t *testing.T) {
|
||||
reached := false
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
reached = true
|
||||
w.WriteHeader(http.StatusCreated)
|
||||
}))
|
||||
defer srv.Close()
|
||||
// httptest listens on 127.0.0.1, which is exactly the destination in question.
|
||||
if !strings.Contains(srv.URL, "127.0.0.1") {
|
||||
t.Skipf("the test server is not on loopback: %s", srv.URL)
|
||||
}
|
||||
|
||||
err := Send(context.Background(), testSubscription(t, srv.URL), Payload{Title: "hej"}, testVAPID(t))
|
||||
if err == nil {
|
||||
t.Fatal("Send reached a loopback address without complaining")
|
||||
}
|
||||
if reached {
|
||||
t.Error("the request arrived: the wall let it through and only the answer failed")
|
||||
}
|
||||
}
|
||||
|
||||
func testSubscription(t *testing.T, endpoint string) Subscription {
|
||||
t.Helper()
|
||||
key, err := ecdh.P256().GenerateKey(rand.Reader)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
auth := make([]byte, 16)
|
||||
if _, err := rand.Read(auth); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return Subscription{
|
||||
Endpoint: endpoint,
|
||||
P256dh: b64.EncodeToString(key.PublicKey().Bytes()),
|
||||
Auth: b64.EncodeToString(auth),
|
||||
}
|
||||
}
|
||||
|
||||
func testVAPID(t *testing.T) VAPID {
|
||||
t.Helper()
|
||||
pub, priv, err := GenerateVAPIDKeys()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return VAPID{Public: pub, Private: priv, Subject: "mailto:[email protected]"}
|
||||
}
|
||||
@@ -0,0 +1,136 @@
|
||||
// Package safedial refuses to open a connection to this machine or to the
|
||||
// network it sits on.
|
||||
//
|
||||
// The problem it solves is the same one three times: somewhere a user supplies a
|
||||
// URL or a host, and the server fetches it. An AI provider's base address, a push
|
||||
// endpoint, an IMAP host, a beacon collector. Each of them is a feature, and each
|
||||
// of them is also a way to ask the server to reach something the caller cannot —
|
||||
// the panel next door, a metadata endpoint, a database on the same bridge — and
|
||||
// to report back whether it answered. On a homelab that is the whole point of the
|
||||
// box being where it is.
|
||||
//
|
||||
// # Why the check is on the address and not on the URL
|
||||
//
|
||||
// Parsing the URL and refusing "localhost" or "127.0.0.1" is the obvious version
|
||||
// and it does not work: a name resolves to whatever its owner says, so
|
||||
// `evil.example.com` can answer 127.0.0.1 — and can answer differently the second
|
||||
// time, after a parse-time check has passed. That is DNS rebinding, and the only
|
||||
// place it can be caught is after resolution and before the connection, which is
|
||||
// exactly where net.Dialer's Control hook runs.
|
||||
//
|
||||
// # What is refused
|
||||
//
|
||||
// Loopback, link-local (including 169.254.169.254, which is where cloud metadata
|
||||
// lives), the private ranges, unique-local IPv6, and the unspecified address.
|
||||
// Everything else is allowed: this is not an allowlist of the internet, it is a
|
||||
// wall around the inside.
|
||||
package safedial
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"syscall"
|
||||
"time"
|
||||
)
|
||||
|
||||
// ErrPrivate is returned when a connection was refused for where it pointed. It
|
||||
// deliberately does not say which address it resolved to: the answer is itself
|
||||
// the thing an attacker is fishing for.
|
||||
type ErrPrivate struct{ Host string }
|
||||
|
||||
func (e *ErrPrivate) Error() string {
|
||||
return fmt.Sprintf("refusing to connect to %s: it resolves to an address on this machine or its private network", e.Host)
|
||||
}
|
||||
|
||||
// Blocked reports whether an address must not be dialled.
|
||||
func Blocked(ip net.IP) bool {
|
||||
return ip == nil ||
|
||||
ip.IsLoopback() ||
|
||||
ip.IsUnspecified() ||
|
||||
ip.IsLinkLocalUnicast() ||
|
||||
ip.IsLinkLocalMulticast() ||
|
||||
ip.IsInterfaceLocalMulticast() ||
|
||||
ip.IsMulticast() ||
|
||||
ip.IsPrivate()
|
||||
}
|
||||
|
||||
// Control is a net.Dialer Control function that refuses a private destination.
|
||||
//
|
||||
// It runs after the name has been resolved and before the socket is connected,
|
||||
// which is the only moment where the address that will actually be used is known.
|
||||
func Control(network, address string, _ syscall.RawConn) error {
|
||||
host, _, err := net.SplitHostPort(address)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
ip := net.ParseIP(host)
|
||||
if Blocked(ip) {
|
||||
return &ErrPrivate{Host: host}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Dialer is a net.Dialer that will not reach inside.
|
||||
func Dialer(timeout time.Duration) *net.Dialer {
|
||||
return &net.Dialer{Timeout: timeout, Control: Control}
|
||||
}
|
||||
|
||||
// Client is an http.Client for fetching something a user named.
|
||||
//
|
||||
// Redirects are followed but capped, and every hop goes through the same dialer —
|
||||
// a redirect to 127.0.0.1 is the oldest way around a check that only looked at the
|
||||
// URL somebody typed.
|
||||
func Client(timeout time.Duration) *http.Client {
|
||||
return &http.Client{
|
||||
Timeout: timeout,
|
||||
Transport: &http.Transport{
|
||||
DialContext: Dialer(timeout).DialContext,
|
||||
TLSHandshakeTimeout: timeout,
|
||||
ResponseHeaderTimeout: timeout,
|
||||
},
|
||||
CheckRedirect: func(req *http.Request, via []*http.Request) error {
|
||||
if len(via) >= 5 {
|
||||
return fmt.Errorf("too many redirects")
|
||||
}
|
||||
return nil
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// AllowPrivate is the same client without the wall, for a destination the
|
||||
// operator configured rather than a user — the panel, and nothing else. Named so
|
||||
// that using it is a decision somebody wrote down.
|
||||
func AllowPrivate(timeout time.Duration) *http.Client {
|
||||
return &http.Client{Timeout: timeout}
|
||||
}
|
||||
|
||||
// CheckURL reports why an address must not be fetched, or "" if it may be.
|
||||
//
|
||||
// A convenience for saying no at the moment somebody types an address, rather
|
||||
// than later when a request quietly fails. It is NOT the protection — a name can
|
||||
// resolve one way now and another way when it is dialled, which is why Control
|
||||
// exists and runs every time. Anything this rejects is certainly wrong; anything
|
||||
// it accepts is merely not obviously wrong.
|
||||
func CheckURL(raw string) string {
|
||||
u, err := url.Parse(raw)
|
||||
if err != nil || u.Host == "" {
|
||||
return "that is not a URL"
|
||||
}
|
||||
host := u.Hostname()
|
||||
if ip := net.ParseIP(host); ip != nil && Blocked(ip) {
|
||||
return "that address is on this machine or its private network"
|
||||
}
|
||||
// A bare name is looked up once so an obvious "localhost" is caught here. A
|
||||
// failure to resolve is not an error: the host may simply not exist yet, and
|
||||
// refusing to save an address because DNS was slow would be its own bug.
|
||||
if ips, err := net.LookupIP(host); err == nil {
|
||||
for _, ip := range ips {
|
||||
if Blocked(ip) {
|
||||
return "that address is on this machine or its private network"
|
||||
}
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
package safedial
|
||||
|
||||
import (
|
||||
"net"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestBlockedCoversTheInside(t *testing.T) {
|
||||
blocked := []string{
|
||||
"127.0.0.1", "::1", "0.0.0.0",
|
||||
// Skyens metadata-endepunkt. Den ene adresse, enhver SSRF sigter efter.
|
||||
"169.254.169.254",
|
||||
"10.0.0.5", "172.16.0.1", "192.168.1.10", "fd00::1", "fe80::1",
|
||||
}
|
||||
for _, s := range blocked {
|
||||
if !Blocked(net.ParseIP(s)) {
|
||||
t.Errorf("Blocked(%s) = false", s)
|
||||
}
|
||||
}
|
||||
for _, s := range []string{"93.184.216.34", "8.8.8.8", "2606:2800:220:1:248:1893:25c8:1946"} {
|
||||
if Blocked(net.ParseIP(s)) {
|
||||
t.Errorf("Blocked(%s) = true, men det er en almindelig adresse", s)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// En httptest-server lytter på loopback, hvilket gør den til den perfekte prøve:
|
||||
// den findes, den svarer, og den må ikke kunne nås.
|
||||
func TestClientRefusesLoopback(t *testing.T) {
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
t.Error("forbindelsen blev oprettet; den skulle være afvist")
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
_, err := Client(3 * time.Second).Get(srv.URL)
|
||||
if err == nil {
|
||||
t.Fatal("en forespørgsel til loopback skal afvises")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAllowPrivateStillReaches(t *testing.T) {
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
resp, err := AllowPrivate(3 * time.Second).Get(srv.URL)
|
||||
if err != nil {
|
||||
t.Fatalf("den skal kunne nå det, operatøren selv har peget på: %v", err)
|
||||
}
|
||||
resp.Body.Close()
|
||||
}
|
||||
@@ -0,0 +1,138 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The collector's side of the beacon: what has reported in, and how many.
|
||||
//
|
||||
// Every method here is about counting installations, never about identifying one.
|
||||
// There is no lookup by anything but the id an installation made up for itself,
|
||||
// and there is nothing to join it against.
|
||||
|
||||
// BeaconStats is the whole of what the collector can say.
|
||||
type BeaconStats struct {
|
||||
Total int `json:"total"`
|
||||
Active7d int `json:"active_7d"`
|
||||
Active30d int `json:"active_30d"`
|
||||
ByVersion map[string]int `json:"by_version"`
|
||||
LastPingAt *time.Time `json:"last_ping_at,omitempty"`
|
||||
}
|
||||
|
||||
// maxBeaconInstalls bounds the table.
|
||||
//
|
||||
// The receiving endpoint is unauthenticated — it has to be, since the caller is a
|
||||
// stranger's installation — and it inserts a row keyed on an id the caller made
|
||||
// up. Anybody can therefore mint ids and grow this table until the volume the
|
||||
// database lives on is full. A ceiling turns that from a disk-fill into a number
|
||||
// that is merely wrong, which is the difference between an outage and an
|
||||
// annoyance. A hundred thousand is far above any plausible real count and far
|
||||
// below anything that matters on disk.
|
||||
const maxBeaconInstalls = 100_000
|
||||
|
||||
// RecordBeacon writes down that an installation is alive, and on which version.
|
||||
//
|
||||
// `first_seen` is kept from the original row on purpose: it is the only thing
|
||||
// that distinguishes an installation that has been running for a year from one
|
||||
// that appeared this morning, and it costs nothing to keep.
|
||||
func (db *DB) RecordBeacon(ctx context.Context, instanceID, version string) error {
|
||||
now := time.Now().Unix()
|
||||
|
||||
// An installation already known keeps reporting whatever the ceiling says: the
|
||||
// cap must never stop counting the real ones, only stop new rows from being
|
||||
// invented without limit.
|
||||
var known int
|
||||
if err := db.QueryRowContext(ctx,
|
||||
`SELECT count(*) FROM beacon_installs WHERE instance_id = ?`, instanceID).Scan(&known); err != nil {
|
||||
return err
|
||||
}
|
||||
if known == 0 {
|
||||
var total int
|
||||
if err := db.QueryRowContext(ctx,
|
||||
`SELECT count(*) FROM beacon_installs`).Scan(&total); err != nil {
|
||||
return err
|
||||
}
|
||||
if total >= maxBeaconInstalls {
|
||||
// Not an error to the caller. Their installation did nothing wrong, and
|
||||
// there is nothing for them to do about it.
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
_, err := db.ExecContext(ctx, `
|
||||
INSERT INTO beacon_installs (instance_id, version, first_seen, last_seen)
|
||||
VALUES (?, ?, ?, ?)
|
||||
ON CONFLICT (instance_id) DO UPDATE SET
|
||||
version = excluded.version,
|
||||
last_seen = excluded.last_seen`,
|
||||
instanceID, version, now, now)
|
||||
return err
|
||||
}
|
||||
|
||||
// BeaconCount is how many installations have reported within the window.
|
||||
//
|
||||
// A window rather than a running total, and that is not a detail. A total only
|
||||
// ever climbs: every instance anybody ever started for ten minutes stays in it
|
||||
// forever, so the number stops meaning "how many people use this" and starts
|
||||
// meaning "how many people have ever tried it". The question being asked is the
|
||||
// first one.
|
||||
func (db *DB) BeaconCount(ctx context.Context, within time.Duration) (int, error) {
|
||||
var n int
|
||||
err := db.QueryRowContext(ctx,
|
||||
`SELECT count(*) FROM beacon_installs WHERE last_seen >= ?`,
|
||||
time.Now().Add(-within).Unix()).Scan(&n)
|
||||
return n, err
|
||||
}
|
||||
|
||||
// BeaconSummary is everything the settings page shows.
|
||||
func (db *DB) BeaconSummary(ctx context.Context) (BeaconStats, error) {
|
||||
out := BeaconStats{ByVersion: map[string]int{}}
|
||||
|
||||
if err := db.QueryRowContext(ctx,
|
||||
`SELECT count(*) FROM beacon_installs`).Scan(&out.Total); err != nil {
|
||||
return out, err
|
||||
}
|
||||
|
||||
var err error
|
||||
if out.Active7d, err = db.BeaconCount(ctx, 7*24*time.Hour); err != nil {
|
||||
return out, err
|
||||
}
|
||||
if out.Active30d, err = db.BeaconCount(ctx, 30*24*time.Hour); err != nil {
|
||||
return out, err
|
||||
}
|
||||
|
||||
// Versions of the ones still alive, not of everything ever seen: an instance
|
||||
// that stopped reporting two years ago should not still be counted as running
|
||||
// the version it was on when it stopped.
|
||||
rows, err := db.QueryContext(ctx, `
|
||||
SELECT version, count(*) FROM beacon_installs
|
||||
WHERE last_seen >= ? AND version <> ''
|
||||
GROUP BY version`, time.Now().Add(-30*24*time.Hour).Unix())
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
defer rows.Close()
|
||||
for rows.Next() {
|
||||
var v string
|
||||
var n int
|
||||
if err := rows.Scan(&v, &n); err != nil {
|
||||
return out, err
|
||||
}
|
||||
out.ByVersion[v] = n
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return out, err
|
||||
}
|
||||
|
||||
var last *int64
|
||||
if err := db.QueryRowContext(ctx,
|
||||
`SELECT max(last_seen) FROM beacon_installs`).Scan(&last); err != nil {
|
||||
return out, err
|
||||
}
|
||||
if last != nil {
|
||||
t := time.Unix(*last, 0)
|
||||
out.LastPingAt = &t
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
@@ -0,0 +1,531 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// CalendarAccount is one connected account belonging to one person.
|
||||
type CalendarAccount struct {
|
||||
ID string `json:"id"`
|
||||
UserID string `json:"-"`
|
||||
Provider string `json:"provider"`
|
||||
Account string `json:"account,omitempty"`
|
||||
|
||||
// URL er tom for en OAuth-konto og adressen for et abonnement. Den er
|
||||
// identiteten på et abonnement, som tokenet er det for Google: to rækker med
|
||||
// den samme adresse for den samme person ville være den samme kalender hentet
|
||||
// to gange, uden nogen måde at sige hvilken der gælder.
|
||||
URL string `json:"url,omitempty"`
|
||||
|
||||
RefreshToken string `json:"-"` // never leaves the server
|
||||
AccessToken string `json:"-"`
|
||||
ExpiresAt time.Time `json:"-"`
|
||||
|
||||
LastSyncAt time.Time `json:"last_sync_at,omitempty"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
}
|
||||
|
||||
// Calendar is one calendar inside an account, shown or not.
|
||||
type Calendar struct {
|
||||
ID string `json:"id"`
|
||||
AccountID string `json:"-"`
|
||||
RemoteID string `json:"remote_id"`
|
||||
Name string `json:"name"`
|
||||
Colour string `json:"colour,omitempty"`
|
||||
TimeZone string `json:"time_zone,omitempty"`
|
||||
Primary bool `json:"primary"`
|
||||
// Writable says the connected account could write to it. Nothing does — the
|
||||
// scope is read-only — and it is stored so the day something can, the interface
|
||||
// knows which calendars could even be offered.
|
||||
Writable bool `json:"writable"`
|
||||
Shown bool `json:"shown"`
|
||||
SortOrder int `json:"-"`
|
||||
}
|
||||
|
||||
// CalendarEvent is one occurrence, in the shape a grid asks about: days at both
|
||||
// ends, inclusive.
|
||||
type CalendarEvent struct {
|
||||
ID string `json:"id"`
|
||||
CalendarID string `json:"calendar_id"`
|
||||
RemoteID string `json:"-"`
|
||||
Summary string `json:"summary"`
|
||||
StartsAt string `json:"starts_at,omitempty"`
|
||||
EndsAt string `json:"ends_at,omitempty"`
|
||||
StartDay string `json:"start_day"`
|
||||
EndDay string `json:"end_day"`
|
||||
AllDay bool `json:"all_day"`
|
||||
Location string `json:"location,omitempty"`
|
||||
URL string `json:"url,omitempty"`
|
||||
|
||||
// CalendarName and Colour ride along with the event rather than being looked up
|
||||
// per row by the caller. Same reasoning as the assignee names on a task list:
|
||||
// the chip has to be drawn in the calendar's own colour, and a second request
|
||||
// per event to learn it is a request per event.
|
||||
CalendarName string `json:"calendar_name,omitempty"`
|
||||
Colour string `json:"colour,omitempty"`
|
||||
}
|
||||
|
||||
const calendarAccountColumns = `id, user_id, provider, account, refresh_token,
|
||||
access_token, expires_at, last_sync_at, created_at`
|
||||
|
||||
// Med adressen. Skrivningen tilføjer den selv, fordi indsættelsen navngiver
|
||||
// søjlerne; læsningen skal bede om den.
|
||||
const calendarAccountRead = calendarAccountColumns + `, url`
|
||||
|
||||
// CalendarAccountFor returns a person's connection to one provider, or nil.
|
||||
func (db *DB) CalendarAccountFor(ctx context.Context, userID, provider string) (*CalendarAccount, error) {
|
||||
// `url = ''`: en OAuth-konto. Uden det ville et abonnement hos den samme
|
||||
// udbyder kunne svare på spørgsmålet "hvad er personen forbundet til med
|
||||
// Google", og et abonnement har hverken token eller konto at svare med.
|
||||
row := db.QueryRowContext(ctx, `SELECT `+calendarAccountRead+`
|
||||
FROM calendar_accounts WHERE user_id = ? AND provider = ? AND url = ''`, userID, provider)
|
||||
a, err := db.scanCalendarAccount(row)
|
||||
if err == sql.ErrNoRows {
|
||||
return nil, nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &a, nil
|
||||
}
|
||||
|
||||
// SaveCalendarAccount writes one, sealing the two columns that must not travel
|
||||
// with a backup. Called by hand rather than by the settings tables' own seal,
|
||||
// which does not reach this table.
|
||||
func (db *DB) SaveCalendarAccount(ctx context.Context, a *CalendarAccount) error {
|
||||
if a.ID == "" {
|
||||
a.ID = NewID()
|
||||
}
|
||||
if a.Provider == "" {
|
||||
a.Provider = "google"
|
||||
}
|
||||
if a.CreatedAt.IsZero() {
|
||||
a.CreatedAt = time.Now()
|
||||
}
|
||||
|
||||
refresh, err := db.sealValue(a.RefreshToken)
|
||||
if err != nil {
|
||||
return fmt.Errorf("seal refresh_token: %w", err)
|
||||
}
|
||||
access, err := db.sealValue(a.AccessToken)
|
||||
if err != nil {
|
||||
return fmt.Errorf("seal access_token: %w", err)
|
||||
}
|
||||
|
||||
// The conflict is on the pair, not on the id: a reconnect builds a fresh
|
||||
// struct with no id in it, and matching on the id would insert a second row
|
||||
// that the unique index then refuses — which reads to the person as "connecting
|
||||
// failed" when what happened is that it already had.
|
||||
// `WHERE url = ''` fordi indekset er delvist siden 0026.
|
||||
//
|
||||
// En OAuth-konto er én pr. person og udbyder — det er tokenet, der er
|
||||
// identiteten — mens et abonnement er én pr. adresse, og man har flere. De to
|
||||
// bor i samme tabel, fordi et abonnement *er* en kalender med begivenheder i,
|
||||
// og den halvdel er den samme. Målet for konflikten skal sige, hvilken af de to
|
||||
// slags rækker der tales om, ellers ved SQLite det ikke.
|
||||
_, err = db.ExecContext(ctx, `
|
||||
INSERT INTO calendar_accounts (`+calendarAccountColumns+`, url)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT (user_id, provider) WHERE url = '' DO UPDATE SET
|
||||
account = excluded.account,
|
||||
refresh_token = excluded.refresh_token,
|
||||
access_token = excluded.access_token,
|
||||
expires_at = excluded.expires_at,
|
||||
last_sync_at = excluded.last_sync_at`,
|
||||
a.ID, a.UserID, a.Provider, a.Account, refresh, access,
|
||||
unixOrZero(a.ExpiresAt), unixOrZero(a.LastSyncAt), a.CreatedAt.Unix(), a.URL)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if a.URL != "" {
|
||||
return nil
|
||||
}
|
||||
|
||||
// The row that is actually there wins. On a reconnect the insert above updated
|
||||
// an existing row, so the id in hand is one nothing points at — and the
|
||||
// calendars are hung off the id, so writing them against it would attach them
|
||||
// to nothing.
|
||||
return db.QueryRowContext(ctx,
|
||||
`SELECT id FROM calendar_accounts WHERE user_id = ? AND provider = ? AND url = ''`,
|
||||
a.UserID, a.Provider).Scan(&a.ID)
|
||||
}
|
||||
|
||||
// MarkCalendarSynced records how far a run got. Written separately from the rest
|
||||
// so a sync never rewrites the credentials it is holding in memory.
|
||||
func (db *DB) MarkCalendarSynced(ctx context.Context, accountID string, at time.Time) error {
|
||||
_, err := db.ExecContext(ctx,
|
||||
`UPDATE calendar_accounts SET last_sync_at = ? WHERE id = ?`, at.Unix(), accountID)
|
||||
return err
|
||||
}
|
||||
|
||||
// DeleteCalendarAccount disconnects one. The calendars and their events go with it
|
||||
// — unlike the tasks a mailbox made, none of this is the person's own work; it is
|
||||
// a copy of somebody else's calendar and it has no meaning without the account.
|
||||
func (db *DB) DeleteCalendarAccount(ctx context.Context, userID, id string) error {
|
||||
_, err := db.ExecContext(ctx,
|
||||
`DELETE FROM calendar_accounts WHERE id = ? AND user_id = ?`, id, userID)
|
||||
return err
|
||||
}
|
||||
|
||||
// UsersWithCalendars returns everybody the sweep has to visit.
|
||||
func (db *DB) UsersWithCalendars(ctx context.Context) ([]string, error) {
|
||||
rows, err := db.QueryContext(ctx, `SELECT DISTINCT user_id FROM calendar_accounts`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var ids []string
|
||||
for rows.Next() {
|
||||
var id string
|
||||
if err := rows.Scan(&id); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
ids = append(ids, id)
|
||||
}
|
||||
return ids, rows.Err()
|
||||
}
|
||||
|
||||
// Calendars returns what one account holds, in the order the provider listed them.
|
||||
func (db *DB) Calendars(ctx context.Context, accountID string) ([]Calendar, error) {
|
||||
rows, err := db.QueryContext(ctx, `
|
||||
SELECT id, account_id, remote_id, name, colour, time_zone, primary_one,
|
||||
writable, shown, sort_order
|
||||
FROM calendars WHERE account_id = ? ORDER BY sort_order, name`, accountID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var out []Calendar
|
||||
for rows.Next() {
|
||||
var c Calendar
|
||||
if err := rows.Scan(&c.ID, &c.AccountID, &c.RemoteID, &c.Name, &c.Colour,
|
||||
&c.TimeZone, &c.Primary, &c.Writable, &c.Shown, &c.SortOrder); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, c)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
// ReplaceCalendars writes the list the provider just gave, keeping what the person
|
||||
// chose.
|
||||
//
|
||||
// The choice survives because it is not in the list: Google says which calendars
|
||||
// exist and what they are called, and only verdande knows which of them somebody
|
||||
// wants to look at. Merged rather than deleted and rewritten, so ticking a calendar
|
||||
// on does not come undone the next time the sweep runs.
|
||||
//
|
||||
// A calendar that has disappeared from the account is removed, and its events go
|
||||
// with it — a calendar somebody was unshared from should stop being drawn, not
|
||||
// stay behind as a frozen copy of what it held on the last good day.
|
||||
func (db *DB) ReplaceCalendars(ctx context.Context, accountID string, list []Calendar) error {
|
||||
return db.Tx(ctx, func(tx *sql.Tx) error {
|
||||
keep := map[string]bool{}
|
||||
for i, c := range list {
|
||||
keep[c.RemoteID] = true
|
||||
// The insert cannot carry `shown`: on conflict it would overwrite the
|
||||
// choice with the default. A new calendar starts hidden — an account
|
||||
// with a dozen shared calendars in it should not fill somebody's grid
|
||||
// the moment they connect it.
|
||||
if _, err := tx.ExecContext(ctx, `
|
||||
INSERT INTO calendars
|
||||
(id, account_id, remote_id, name, colour, time_zone, primary_one,
|
||||
writable, shown, sort_order)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT (account_id, remote_id) DO UPDATE SET
|
||||
name = excluded.name,
|
||||
colour = excluded.colour,
|
||||
time_zone = excluded.time_zone,
|
||||
primary_one = excluded.primary_one,
|
||||
writable = excluded.writable,
|
||||
sort_order = excluded.sort_order`,
|
||||
NewID(), accountID, c.RemoteID, c.Name, c.Colour, c.TimeZone,
|
||||
c.Primary, c.Writable, c.Shown, i); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
rows, err := tx.QueryContext(ctx,
|
||||
`SELECT id, remote_id FROM calendars WHERE account_id = ?`, accountID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var gone []string
|
||||
for rows.Next() {
|
||||
var id, remote string
|
||||
if err := rows.Scan(&id, &remote); err != nil {
|
||||
rows.Close()
|
||||
return err
|
||||
}
|
||||
if !keep[remote] {
|
||||
gone = append(gone, id)
|
||||
}
|
||||
}
|
||||
// Closed before the next statement runs on this transaction. An open result
|
||||
// set with a query behind it is the deadlock the pool comment in store.go
|
||||
// exists to make slow rather than fatal.
|
||||
rows.Close()
|
||||
if err := rows.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for _, id := range gone {
|
||||
if _, err := tx.ExecContext(ctx, `DELETE FROM calendars WHERE id = ?`, id); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
// ShowCalendars sets which of an account's calendars are drawn.
|
||||
//
|
||||
// The whole set in one write rather than one flip at a time, for the reason the
|
||||
// projects' order is written the same way: a person has a handful of calendars, the
|
||||
// question is "these ones", and a list that can only be built by several requests
|
||||
// can land half applied.
|
||||
func (db *DB) ShowCalendars(ctx context.Context, accountID string, shown []string) error {
|
||||
return db.Tx(ctx, func(tx *sql.Tx) error {
|
||||
if _, err := tx.ExecContext(ctx,
|
||||
`UPDATE calendars SET shown = 0 WHERE account_id = ?`, accountID); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, id := range shown {
|
||||
// Scoped in the SQL rather than checked first: an id read in one step
|
||||
// and written in the next is a window, and the account is the whole of
|
||||
// the permission here.
|
||||
if _, err := tx.ExecContext(ctx,
|
||||
`UPDATE calendars SET shown = 1 WHERE id = ? AND account_id = ?`,
|
||||
id, accountID); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
// Events of a calendar nobody is looking at are dead weight in a table the
|
||||
// grid queries. They come back with the next sync if it is turned on again.
|
||||
_, err := tx.ExecContext(ctx, `
|
||||
DELETE FROM calendar_events WHERE calendar_id IN
|
||||
(SELECT id FROM calendars WHERE account_id = ? AND shown = 0)`, accountID)
|
||||
return err
|
||||
})
|
||||
}
|
||||
|
||||
// ReplaceCalendarEvents swaps one calendar's events inside a window.
|
||||
//
|
||||
// Replaced rather than merged, and no sync token. Google offers incremental sync,
|
||||
// and it would save bytes verdande is not short of while costing the one thing it
|
||||
// cannot afford: a token that has to be right, forever, for the grid not to be
|
||||
// quietly wrong. A window that is deleted and rewritten cannot drift — an event
|
||||
// somebody moved out of the window disappears because it is not in the answer, not
|
||||
// because verdande remembered to delete it.
|
||||
//
|
||||
// Bounded by the window at both ends, so a sync of September does not throw away
|
||||
// August.
|
||||
func (db *DB) ReplaceCalendarEvents(ctx context.Context, calendarID, from, to string, events []CalendarEvent) error {
|
||||
return db.Tx(ctx, func(tx *sql.Tx) error {
|
||||
if _, err := tx.ExecContext(ctx, `
|
||||
DELETE FROM calendar_events
|
||||
WHERE calendar_id = ? AND end_day >= ? AND start_day <= ?`,
|
||||
calendarID, from, to); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, e := range events {
|
||||
if _, err := tx.ExecContext(ctx, `
|
||||
INSERT INTO calendar_events
|
||||
(id, calendar_id, remote_id, summary, starts_at, ends_at,
|
||||
start_day, end_day, all_day, location, url)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
||||
NewID(), calendarID, e.RemoteID, e.Summary, e.StartsAt, e.EndsAt,
|
||||
e.StartDay, e.EndDay, e.AllDay, e.Location, e.URL); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
// CalendarEvents returns everything one person can see between two days,
|
||||
// inclusive at both ends.
|
||||
//
|
||||
// Scoped to the caller in the join rather than checked afterwards: the events of
|
||||
// somebody else's calendar are somebody else's, and a filter applied after the read
|
||||
// is a filter that can be forgotten.
|
||||
func (db *DB) CalendarEvents(ctx context.Context, userID, from, to string) ([]CalendarEvent, error) {
|
||||
rows, err := db.QueryContext(ctx, `
|
||||
SELECT e.id, e.calendar_id, e.remote_id, e.summary, e.starts_at, e.ends_at,
|
||||
e.start_day, e.end_day, e.all_day, e.location, e.url, c.name, c.colour
|
||||
FROM calendar_events e
|
||||
JOIN calendars c ON c.id = e.calendar_id
|
||||
JOIN calendar_accounts a ON a.id = c.account_id
|
||||
WHERE a.user_id = ? AND c.shown = 1 AND e.end_day >= ? AND e.start_day <= ?
|
||||
ORDER BY e.start_day, e.all_day DESC, e.starts_at`, userID, from, to)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var out []CalendarEvent
|
||||
for rows.Next() {
|
||||
var e CalendarEvent
|
||||
if err := rows.Scan(&e.ID, &e.CalendarID, &e.RemoteID, &e.Summary, &e.StartsAt,
|
||||
&e.EndsAt, &e.StartDay, &e.EndDay, &e.AllDay, &e.Location, &e.URL,
|
||||
&e.CalendarName, &e.Colour); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, e)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
func (db *DB) scanCalendarAccount(row scanner) (CalendarAccount, error) {
|
||||
var a CalendarAccount
|
||||
var expires, lastSync, created int64
|
||||
err := row.Scan(&a.ID, &a.UserID, &a.Provider, &a.Account, &a.RefreshToken,
|
||||
&a.AccessToken, &expires, &lastSync, &created, &a.URL)
|
||||
if err != nil {
|
||||
return a, err
|
||||
}
|
||||
for _, field := range []*string{&a.RefreshToken, &a.AccessToken} {
|
||||
plain, err := db.unsealValue(*field)
|
||||
if err != nil {
|
||||
return a, err
|
||||
}
|
||||
*field = plain
|
||||
}
|
||||
a.ExpiresAt = timeOrZero(expires)
|
||||
a.LastSyncAt = timeOrZero(lastSync)
|
||||
a.CreatedAt = time.Unix(created, 0)
|
||||
return a, nil
|
||||
}
|
||||
|
||||
// --- abonnementer ---------------------------------------------------------------
|
||||
|
||||
// SubscribeCalendar adds a calendar somebody only has an address to.
|
||||
//
|
||||
// One row in `calendar_accounts` with a url and no tokens, and one row in
|
||||
// `calendars` hung off it — because a subscription *is* one calendar, where an
|
||||
// OAuth account is a list of them somebody picks from. Written together so a
|
||||
// subscription is never a row with nothing under it: the sweep looks for accounts,
|
||||
// and one without a calendar would be fetched for ever and stored nowhere.
|
||||
//
|
||||
// Shown from the start. Ticking a box after adding an address is a second step for
|
||||
// a decision the person already made by adding it.
|
||||
func (db *DB) SubscribeCalendar(ctx context.Context, userID, url, name string) (*CalendarAccount, error) {
|
||||
account := &CalendarAccount{
|
||||
ID: NewID(),
|
||||
UserID: userID,
|
||||
Provider: "ics",
|
||||
Account: name,
|
||||
URL: url,
|
||||
CreatedAt: time.Now(),
|
||||
}
|
||||
|
||||
err := db.Tx(ctx, func(tx *sql.Tx) error {
|
||||
if _, err := tx.ExecContext(ctx, `
|
||||
INSERT INTO calendar_accounts (id, user_id, provider, account, url, created_at)
|
||||
VALUES (?, ?, 'ics', ?, ?, ?)`,
|
||||
account.ID, userID, name, url, account.CreatedAt.Unix()); err != nil {
|
||||
return err
|
||||
}
|
||||
_, err := tx.ExecContext(ctx, `
|
||||
INSERT INTO calendars (id, account_id, remote_id, name, shown)
|
||||
VALUES (?, ?, ?, ?, 1)`,
|
||||
NewID(), account.ID, url, name)
|
||||
return err
|
||||
})
|
||||
if isUniqueViolation(err) {
|
||||
return nil, ErrDuplicate
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return account, nil
|
||||
}
|
||||
|
||||
// Subscriptions lists the calendars a person subscribes to by address.
|
||||
func (db *DB) Subscriptions(ctx context.Context, userID string) ([]CalendarAccount, error) {
|
||||
rows, err := db.QueryContext(ctx, `SELECT `+calendarAccountRead+`
|
||||
FROM calendar_accounts WHERE user_id = ? AND url <> ''
|
||||
ORDER BY created_at`, userID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
out := []CalendarAccount{}
|
||||
for rows.Next() {
|
||||
a, err := db.scanCalendarAccount(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, a)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
// AllSubscriptions is every subscription on the instance, for the sweep.
|
||||
//
|
||||
// The sweep runs for the server rather than for a session, so it cannot ask "whose
|
||||
// is this" the way a handler can — it is handed the user id along with the row and
|
||||
// writes against that.
|
||||
func (db *DB) AllSubscriptions(ctx context.Context) ([]CalendarAccount, error) {
|
||||
rows, err := db.QueryContext(ctx, `SELECT `+calendarAccountRead+`
|
||||
FROM calendar_accounts WHERE url <> '' ORDER BY last_sync_at`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
out := []CalendarAccount{}
|
||||
for rows.Next() {
|
||||
a, err := db.scanCalendarAccount(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, a)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
// RenameSubscribedCalendar writes the name the file calls itself.
|
||||
//
|
||||
// A subscription is added with the address as its name, because that is all there
|
||||
// is to go on before it has been fetched once. The file usually says better —
|
||||
// X-WR-CALNAME is what every publisher uses — and taking it means the list says
|
||||
// "Danske helligdage" instead of a URL nobody reads.
|
||||
//
|
||||
// Only when the person has not renamed it themselves: a name they typed is an
|
||||
// instruction, and a name from the file is a guess.
|
||||
func (db *DB) RenameSubscribedCalendar(ctx context.Context, accountID, name string) error {
|
||||
if strings.TrimSpace(name) == "" {
|
||||
return nil
|
||||
}
|
||||
return db.Tx(ctx, func(tx *sql.Tx) error {
|
||||
if _, err := tx.ExecContext(ctx, `
|
||||
UPDATE calendar_accounts SET account = ?
|
||||
WHERE id = ? AND (account = '' OR account = url)`, name, accountID); err != nil {
|
||||
return err
|
||||
}
|
||||
_, err := tx.ExecContext(ctx, `
|
||||
UPDATE calendars SET name = ?
|
||||
WHERE account_id = ? AND (name = '' OR name = remote_id)`, name, accountID)
|
||||
return err
|
||||
})
|
||||
}
|
||||
|
||||
// SubscribedCalendarID is the single calendar row under a subscription.
|
||||
func (db *DB) SubscribedCalendarID(ctx context.Context, accountID string) (string, error) {
|
||||
var id string
|
||||
err := db.QueryRowContext(ctx,
|
||||
`SELECT id FROM calendars WHERE account_id = ? LIMIT 1`, accountID).Scan(&id)
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return "", ErrNotFound
|
||||
}
|
||||
return id, err
|
||||
}
|
||||
@@ -0,0 +1,358 @@
|
||||
package store
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func connectedAccount(t *testing.T, db *DB, userID string) *CalendarAccount {
|
||||
t.Helper()
|
||||
a := &CalendarAccount{
|
||||
UserID: userID, Provider: "google", Account: "[email protected]",
|
||||
RefreshToken: "1//0-a-real-looking-refresh-token", AccessToken: "ya29.access",
|
||||
ExpiresAt: time.Now().Add(time.Hour),
|
||||
}
|
||||
if err := db.SaveCalendarAccount(context.Background(), a); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return a
|
||||
}
|
||||
|
||||
// The same claim the mailbox tokens have to meet: what is written must not be
|
||||
// readable in the file a backup copies.
|
||||
func TestCalendarTokensAreNotStoredInTheClear(t *testing.T) {
|
||||
db, userID := sealedStore(t)
|
||||
ctx := context.Background()
|
||||
a := connectedAccount(t, db, userID)
|
||||
|
||||
var refresh, access string
|
||||
if err := db.QueryRowContext(ctx,
|
||||
`SELECT refresh_token, access_token FROM calendar_accounts WHERE id = ?`,
|
||||
a.ID).Scan(&refresh, &access); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if strings.Contains(refresh, "real-looking") || strings.Contains(access, "ya29") {
|
||||
t.Fatalf("a token is in the row as written: %q %q", refresh, access)
|
||||
}
|
||||
// The address stays readable, which is the reason for sealing named columns
|
||||
// rather than the whole row.
|
||||
var account string
|
||||
if err := db.QueryRowContext(ctx,
|
||||
`SELECT account FROM calendar_accounts WHERE id = ?`, a.ID).Scan(&account); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if account != "[email protected]" {
|
||||
t.Errorf("the address was sealed too: %q", account)
|
||||
}
|
||||
|
||||
back, err := db.CalendarAccountFor(ctx, userID, "google")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if back.RefreshToken != a.RefreshToken || back.AccessToken != a.AccessToken {
|
||||
t.Errorf("the tokens did not survive the round trip: %+v", back)
|
||||
}
|
||||
}
|
||||
|
||||
// Connecting again must replace what was there, not fail. It is the same OAuth
|
||||
// flow run a second time, and the person's word for it is "connect", not "add".
|
||||
func TestConnectingAgainReplacesTheAccount(t *testing.T) {
|
||||
db, userID := sealedStore(t)
|
||||
ctx := context.Background()
|
||||
first := connectedAccount(t, db, userID)
|
||||
|
||||
second := &CalendarAccount{
|
||||
UserID: userID, Provider: "google", Account: "[email protected]",
|
||||
RefreshToken: "1//0-fresh", AccessToken: "ya29.fresh",
|
||||
}
|
||||
if err := db.SaveCalendarAccount(ctx, second); err != nil {
|
||||
t.Fatalf("connecting a second time failed: %v", err)
|
||||
}
|
||||
// And the id in hand is the row that is actually there. The calendars hang off
|
||||
// it, so an id nothing points at would attach them to nothing.
|
||||
if second.ID != first.ID {
|
||||
t.Errorf("the second connect came back with id %q, want the existing %q",
|
||||
second.ID, first.ID)
|
||||
}
|
||||
|
||||
back, err := db.CalendarAccountFor(ctx, userID, "google")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if back.RefreshToken != "1//0-fresh" {
|
||||
t.Errorf("the token came back as %q", back.RefreshToken)
|
||||
}
|
||||
}
|
||||
|
||||
// Google says which calendars exist; only verdande knows which ones somebody wants
|
||||
// to look at. A refresh that came back with the same list must not undo the choice.
|
||||
func TestRefreshingTheListKeepsWhatWasChosen(t *testing.T) {
|
||||
db, userID := sealedStore(t)
|
||||
ctx := context.Background()
|
||||
a := connectedAccount(t, db, userID)
|
||||
|
||||
list := []Calendar{
|
||||
{RemoteID: "primary", Name: "[email protected]", Colour: "#3f51b5", Primary: true},
|
||||
{RemoteID: "fam", Name: "Familien", Colour: "#0b8043"},
|
||||
{RemoteID: "old", Name: "Et delt"},
|
||||
}
|
||||
if err := db.ReplaceCalendars(ctx, a.ID, list); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// Nothing is shown to begin with: an account with a dozen shared calendars in
|
||||
// it should not fill somebody's grid the moment they connect it.
|
||||
stored, err := db.Calendars(ctx, a.ID)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(stored) != 3 {
|
||||
t.Fatalf("got %d calendars, want 3", len(stored))
|
||||
}
|
||||
for _, c := range stored {
|
||||
if c.Shown {
|
||||
t.Errorf("%q was shown without anybody asking for it", c.Name)
|
||||
}
|
||||
}
|
||||
|
||||
var famID string
|
||||
for _, c := range stored {
|
||||
if c.RemoteID == "fam" {
|
||||
famID = c.ID
|
||||
}
|
||||
}
|
||||
if err := db.ShowCalendars(ctx, a.ID, []string{famID}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// The sweep runs again, and one calendar has been renamed and one has gone.
|
||||
if err := db.ReplaceCalendars(ctx, a.ID, []Calendar{
|
||||
{RemoteID: "primary", Name: "[email protected]", Colour: "#3f51b5", Primary: true},
|
||||
{RemoteID: "fam", Name: "Familien og hunden", Colour: "#0b8043"},
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
stored, err = db.Calendars(ctx, a.ID)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(stored) != 2 {
|
||||
t.Fatalf("got %d calendars; one that has left the account should stop being drawn",
|
||||
len(stored))
|
||||
}
|
||||
for _, c := range stored {
|
||||
switch c.RemoteID {
|
||||
case "fam":
|
||||
if !c.Shown {
|
||||
t.Error("the choice was undone by a refresh of the list")
|
||||
}
|
||||
if c.Name != "Familien og hunden" {
|
||||
t.Errorf("the new name did not arrive: %q", c.Name)
|
||||
}
|
||||
if c.ID != famID {
|
||||
t.Error("the row was rebuilt rather than updated; its id is what was chosen")
|
||||
}
|
||||
case "primary":
|
||||
if c.Shown {
|
||||
t.Error("a calendar nobody chose became shown")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A window is replaced, not merged. An event moved out of it has to disappear
|
||||
// because it is absent from the answer — and the months on either side must not go
|
||||
// with it.
|
||||
func TestReplacingAWindowLeavesTheMonthsAroundItAlone(t *testing.T) {
|
||||
db, userID := sealedStore(t)
|
||||
ctx := context.Background()
|
||||
a := connectedAccount(t, db, userID)
|
||||
|
||||
if err := db.ReplaceCalendars(ctx, a.ID, []Calendar{{RemoteID: "primary", Name: "Min"}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
stored, _ := db.Calendars(ctx, a.ID)
|
||||
cal := stored[0]
|
||||
if err := db.ShowCalendars(ctx, a.ID, []string{cal.ID}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
july := CalendarEvent{RemoteID: "j", Summary: "Juli", StartDay: "2026-07-15", EndDay: "2026-07-15"}
|
||||
august := CalendarEvent{RemoteID: "a", Summary: "August", StartDay: "2026-08-15", EndDay: "2026-08-15"}
|
||||
if err := db.ReplaceCalendarEvents(ctx, cal.ID, "2026-07-01", "2026-07-31",
|
||||
[]CalendarEvent{july}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.ReplaceCalendarEvents(ctx, cal.ID, "2026-08-01", "2026-08-31",
|
||||
[]CalendarEvent{august}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
got, err := db.CalendarEvents(ctx, userID, "2026-07-01", "2026-08-31")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(got) != 2 {
|
||||
t.Fatalf("got %d events, want both months: %+v", len(got), got)
|
||||
}
|
||||
|
||||
// August is synced again and the meeting has been cancelled. July stays.
|
||||
if err := db.ReplaceCalendarEvents(ctx, cal.ID, "2026-08-01", "2026-08-31", nil); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err = db.CalendarEvents(ctx, userID, "2026-07-01", "2026-08-31")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(got) != 1 || got[0].Summary != "Juli" {
|
||||
t.Fatalf("got %+v; syncing August threw away July", got)
|
||||
}
|
||||
}
|
||||
|
||||
// The grid asks "which cells does this cover", so an event that straddles the edge
|
||||
// of the question has to come back.
|
||||
func TestAnEventThatStraddlesTheWindowIsFound(t *testing.T) {
|
||||
db, userID := sealedStore(t)
|
||||
ctx := context.Background()
|
||||
a := connectedAccount(t, db, userID)
|
||||
if err := db.ReplaceCalendars(ctx, a.ID, []Calendar{{RemoteID: "c", Name: "Min", Colour: "#0b8043"}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
stored, _ := db.Calendars(ctx, a.ID)
|
||||
if err := db.ShowCalendars(ctx, a.ID, []string{stored[0].ID}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.ReplaceCalendarEvents(ctx, stored[0].ID, "2026-08-01", "2026-09-30",
|
||||
[]CalendarEvent{
|
||||
{RemoteID: "trip", Summary: "Aarhus", StartDay: "2026-08-28", EndDay: "2026-09-02", AllDay: true},
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// A week that contains neither end of it.
|
||||
got, err := db.CalendarEvents(ctx, userID, "2026-08-31", "2026-09-01")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(got) != 1 {
|
||||
t.Fatalf("got %+v; an event covering the whole week was not found", got)
|
||||
}
|
||||
// The calendar's own colour rides along, or the chip has to ask for it per row.
|
||||
if got[0].Colour != "#0b8043" || got[0].CalendarName != "Min" {
|
||||
t.Errorf("the calendar did not come with the event: %+v", got[0])
|
||||
}
|
||||
}
|
||||
|
||||
// Somebody else's calendar is somebody else's.
|
||||
func TestOnlyTheOwnersEventsComeBack(t *testing.T) {
|
||||
db, userID := sealedStore(t)
|
||||
ctx := context.Background()
|
||||
|
||||
other := &User{Email: "[email protected]", Name: "Andreas"}
|
||||
if err := db.CreateUser(ctx, other, "Indbakke"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
theirs := &CalendarAccount{UserID: other.ID, Provider: "google", RefreshToken: "r"}
|
||||
if err := db.SaveCalendarAccount(ctx, theirs); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.ReplaceCalendars(ctx, theirs.ID, []Calendar{{RemoteID: "c", Name: "Deres"}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
stored, _ := db.Calendars(ctx, theirs.ID)
|
||||
if err := db.ShowCalendars(ctx, theirs.ID, []string{stored[0].ID}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.ReplaceCalendarEvents(ctx, stored[0].ID, "2026-08-01", "2026-08-31",
|
||||
[]CalendarEvent{{RemoteID: "x", Summary: "Privat", StartDay: "2026-08-10", EndDay: "2026-08-10"}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
got, err := db.CalendarEvents(ctx, userID, "2026-08-01", "2026-08-31")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(got) != 0 {
|
||||
t.Fatalf("somebody else's calendar came back: %+v", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A calendar turned off stops being drawn, and stops costing the grid anything.
|
||||
func TestTurningACalendarOffRemovesItsEvents(t *testing.T) {
|
||||
db, userID := sealedStore(t)
|
||||
ctx := context.Background()
|
||||
a := connectedAccount(t, db, userID)
|
||||
if err := db.ReplaceCalendars(ctx, a.ID, []Calendar{{RemoteID: "c", Name: "Min"}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
stored, _ := db.Calendars(ctx, a.ID)
|
||||
if err := db.ShowCalendars(ctx, a.ID, []string{stored[0].ID}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.ReplaceCalendarEvents(ctx, stored[0].ID, "2026-08-01", "2026-08-31",
|
||||
[]CalendarEvent{{RemoteID: "x", Summary: "Møde", StartDay: "2026-08-10", EndDay: "2026-08-10"}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if err := db.ShowCalendars(ctx, a.ID, nil); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var count int
|
||||
if err := db.QueryRowContext(ctx, `SELECT count(*) FROM calendar_events`).Scan(&count); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if count != 0 {
|
||||
t.Errorf("%d events of a calendar nobody is looking at are still in the table", count)
|
||||
}
|
||||
}
|
||||
|
||||
// Disconnecting takes the calendars and the events with it. Unlike the tasks a
|
||||
// mailbox made, none of this is the person's own work — it is a copy of somebody
|
||||
// else's calendar and it means nothing without the account.
|
||||
func TestDisconnectingTakesEverythingWithIt(t *testing.T) {
|
||||
db, userID := sealedStore(t)
|
||||
ctx := context.Background()
|
||||
a := connectedAccount(t, db, userID)
|
||||
if err := db.ReplaceCalendars(ctx, a.ID, []Calendar{{RemoteID: "c", Name: "Min"}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
stored, _ := db.Calendars(ctx, a.ID)
|
||||
if err := db.ShowCalendars(ctx, a.ID, []string{stored[0].ID}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.ReplaceCalendarEvents(ctx, stored[0].ID, "2026-08-01", "2026-08-31",
|
||||
[]CalendarEvent{{RemoteID: "x", Summary: "Møde", StartDay: "2026-08-10", EndDay: "2026-08-10"}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if err := db.DeleteCalendarAccount(ctx, userID, a.ID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, table := range []string{"calendar_accounts", "calendars", "calendar_events"} {
|
||||
var count int
|
||||
if err := db.QueryRowContext(ctx, `SELECT count(*) FROM `+table).Scan(&count); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if count != 0 {
|
||||
t.Errorf("%s still holds %d rows after disconnecting", table, count)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The sweep finds people by the presence of an account, so it has to keep working
|
||||
// now that the credential in it is ciphertext.
|
||||
func TestTheSweepFindsAConnectedCalendar(t *testing.T) {
|
||||
db, userID := sealedStore(t)
|
||||
connectedAccount(t, db, userID)
|
||||
|
||||
users, err := db.UsersWithCalendars(context.Background())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(users) != 1 || users[0] != userID {
|
||||
t.Fatalf("the sweep found %v", users)
|
||||
}
|
||||
}
|
||||
@@ -27,6 +27,7 @@ type Attachment struct {
|
||||
// GroupID hangs the file on a project group rather than on any one task in
|
||||
// it: the contract that governs all of "Arbejde", not a step in it.
|
||||
GroupID string
|
||||
NoteID string
|
||||
Filename string
|
||||
MimeType string
|
||||
Size int64
|
||||
@@ -157,17 +158,18 @@ func (db *DB) CreateAttachment(ctx context.Context, a *Attachment) error {
|
||||
a.CreatedAt = time.Now().UTC()
|
||||
|
||||
_, err := db.ExecContext(ctx,
|
||||
`INSERT INTO attachments (id, task_id, comment_id, group_id, filename, mime_type,
|
||||
`INSERT INTO attachments (id, task_id, comment_id, group_id, note_id, filename, mime_type,
|
||||
size, path, uploaded_by, created_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
||||
a.ID, nullString(a.TaskID), nullString(a.CommentID), nullString(a.GroupID),
|
||||
nullString(a.NoteID),
|
||||
a.Filename, a.MimeType, a.Size, a.Path, a.UploadedBy, a.CreatedAt.Unix())
|
||||
return err
|
||||
}
|
||||
|
||||
func (db *DB) ListTaskAttachments(ctx context.Context, taskID string) ([]Attachment, error) {
|
||||
rows, err := db.QueryContext(ctx,
|
||||
`SELECT id, task_id, comment_id, group_id, filename, mime_type, size, path, uploaded_by, created_at
|
||||
`SELECT id, task_id, comment_id, group_id, note_id, filename, mime_type, size, path, uploaded_by, created_at
|
||||
FROM attachments WHERE task_id = ? ORDER BY created_at`, taskID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -176,11 +178,24 @@ func (db *DB) ListTaskAttachments(ctx context.Context, taskID string) ([]Attachm
|
||||
return scanAttachments(rows)
|
||||
}
|
||||
|
||||
// ListNoteAttachments is the files inside one note — the pictures and scans that
|
||||
// came with it, and anything added since.
|
||||
func (db *DB) ListNoteAttachments(ctx context.Context, noteID string) ([]Attachment, error) {
|
||||
rows, err := db.QueryContext(ctx,
|
||||
`SELECT id, task_id, comment_id, group_id, note_id, filename, mime_type, size, path, uploaded_by, created_at
|
||||
FROM attachments WHERE note_id = ? ORDER BY created_at`, noteID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
return scanAttachments(rows)
|
||||
}
|
||||
|
||||
// ListGroupAttachments is the documents that belong to a whole group rather than
|
||||
// to anything inside it.
|
||||
func (db *DB) ListGroupAttachments(ctx context.Context, groupID string) ([]Attachment, error) {
|
||||
rows, err := db.QueryContext(ctx,
|
||||
`SELECT id, task_id, comment_id, group_id, filename, mime_type, size, path, uploaded_by, created_at
|
||||
`SELECT id, task_id, comment_id, group_id, note_id, filename, mime_type, size, path, uploaded_by, created_at
|
||||
FROM attachments WHERE group_id = ? ORDER BY created_at`, groupID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -195,7 +210,7 @@ func (db *DB) attachmentsByComment(ctx context.Context, commentIDs []string) (ma
|
||||
args[i] = id
|
||||
}
|
||||
rows, err := db.QueryContext(ctx,
|
||||
`SELECT id, task_id, comment_id, group_id, filename, mime_type, size, path, uploaded_by, created_at
|
||||
`SELECT id, task_id, comment_id, group_id, note_id, filename, mime_type, size, path, uploaded_by, created_at
|
||||
FROM attachments WHERE comment_id IN (`+placeholders(len(commentIDs))+`)
|
||||
ORDER BY created_at`, args...)
|
||||
if err != nil {
|
||||
@@ -218,15 +233,16 @@ func scanAttachments(rows *sql.Rows) ([]Attachment, error) {
|
||||
out := []Attachment{}
|
||||
for rows.Next() {
|
||||
var a Attachment
|
||||
var taskID, commentID, groupID sql.NullString
|
||||
var taskID, commentID, groupID, noteID sql.NullString
|
||||
var created int64
|
||||
if err := rows.Scan(&a.ID, &taskID, &commentID, &groupID, &a.Filename, &a.MimeType,
|
||||
if err := rows.Scan(&a.ID, &taskID, &commentID, &groupID, ¬eID, &a.Filename, &a.MimeType,
|
||||
&a.Size, &a.Path, &a.UploadedBy, &created); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
a.TaskID = taskID.String
|
||||
a.CommentID = commentID.String
|
||||
a.GroupID = groupID.String
|
||||
a.NoteID = noteID.String
|
||||
a.CreatedAt = time.Unix(created, 0).UTC()
|
||||
out = append(out, a)
|
||||
}
|
||||
@@ -235,7 +251,7 @@ func scanAttachments(rows *sql.Rows) ([]Attachment, error) {
|
||||
|
||||
func (db *DB) GetAttachment(ctx context.Context, attachmentID string) (*Attachment, error) {
|
||||
rows, err := db.QueryContext(ctx,
|
||||
`SELECT id, task_id, comment_id, group_id, filename, mime_type, size, path, uploaded_by, created_at
|
||||
`SELECT id, task_id, comment_id, group_id, note_id, filename, mime_type, size, path, uploaded_by, created_at
|
||||
FROM attachments WHERE id = ?`, attachmentID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
|
||||
@@ -195,10 +195,14 @@ func TestAGmailConnectedBeforeTheMoveSurvivesIt(t *testing.T) {
|
||||
}
|
||||
values := `{"refresh_token":"` + sealed + `","trigger":"label","label":"Verdande",` +
|
||||
`"email":"[email protected]","seen":["abc","def"],"project_id":"p-1"}`
|
||||
// The conflict target is (user_id, scope) since 0025 — before that the table
|
||||
// held one row per person and the upsert renamed its scope, which is the bug
|
||||
// 0025 exists to undo. Nothing conflicts here; the clause only has to be legal
|
||||
// against the schema this test is running on.
|
||||
if _, err := db.ExecContext(context.Background(), `
|
||||
INSERT INTO user_settings (user_id, scope, values_json, updated_at)
|
||||
VALUES (?, 'gmail', ?, 0)
|
||||
ON CONFLICT (user_id) DO UPDATE SET scope = 'gmail', values_json = excluded.values_json`,
|
||||
ON CONFLICT (user_id, scope) DO UPDATE SET values_json = excluded.values_json`,
|
||||
userID, values); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,13 @@
|
||||
-- The order of the fixed views in the sidebar.
|
||||
--
|
||||
-- On the account rather than in the browser, and for the reason the sidebar's
|
||||
-- width is the other way round: a width is a fact about the screen you are at, and
|
||||
-- the answer on a phone is not the answer on a monitor. Which order you want I dag,
|
||||
-- Kommende, Venter and Indbakken in is a fact about you, and it should be the same
|
||||
-- on the laptop as on the phone.
|
||||
--
|
||||
-- A list of keys rather than a number per row, for the same reason sidebar_collapsed
|
||||
-- is: the set of views changes with the program, and a key that no longer exists
|
||||
-- costs nothing — the sidebar draws what it recognises and appends anything new at
|
||||
-- the end. That is what makes adding a view later not a migration.
|
||||
ALTER TABLE users ADD COLUMN nav_order TEXT NOT NULL DEFAULT '[]';
|
||||
@@ -0,0 +1,29 @@
|
||||
-- #garageristeriet and #GarageRisteriet are the same project.
|
||||
--
|
||||
-- They were not, because the key was stored exactly as typed and looked up the
|
||||
-- same way — so a note that named a project in lower case simply did not appear
|
||||
-- on it, silently and with nothing to suggest why.
|
||||
--
|
||||
-- Folded on the way in from now on; this brings the rows already written into
|
||||
-- line. Only projects: a [[note title]] is a title, and two notes called "Møde"
|
||||
-- and "møde" are two notes.
|
||||
--
|
||||
-- The primary key is (note_id, kind, target_id), so folding can collide with a row
|
||||
-- that is already there — a note that wrote both spellings. INSERT OR IGNORE into
|
||||
-- a fresh set and swap, which keeps one of each rather than failing on the pair.
|
||||
CREATE TABLE note_links_folded (
|
||||
note_id TEXT NOT NULL REFERENCES notes (id) ON DELETE CASCADE,
|
||||
kind TEXT NOT NULL CHECK (kind IN ('task', 'note', 'project')),
|
||||
target_id TEXT NOT NULL,
|
||||
PRIMARY KEY (note_id, kind, target_id)
|
||||
);
|
||||
|
||||
INSERT OR IGNORE INTO note_links_folded (note_id, kind, target_id)
|
||||
SELECT note_id, kind,
|
||||
CASE WHEN kind = 'project' THEN lower(target_id) ELSE target_id END
|
||||
FROM note_links;
|
||||
|
||||
DROP TABLE note_links;
|
||||
ALTER TABLE note_links_folded RENAME TO note_links;
|
||||
|
||||
CREATE INDEX note_links_backwards ON note_links (kind, target_id);
|
||||
@@ -0,0 +1,45 @@
|
||||
-- A note can carry files, the way a task and a group already can.
|
||||
--
|
||||
-- Apple Notes is full of photographs, scans and pasted screenshots, and a note
|
||||
-- that arrives here as words alone is not that note — it is a summary of it. The
|
||||
-- import writes the files as attachments and rewrites the links in the text, so a
|
||||
-- picture that was in the middle of a paragraph is still in the middle of it.
|
||||
--
|
||||
-- Same table as everything else attaches to, rather than one of its own. The files
|
||||
-- are content-addressed and deduplicated on disk, the upload path is written and
|
||||
-- tested, and the cleanup that removes an orphan already walks this table. A
|
||||
-- second table would be a second everything.
|
||||
|
||||
-- verdande:rebuild-tables
|
||||
|
||||
CREATE TABLE attachments_with_notes (
|
||||
id TEXT PRIMARY KEY,
|
||||
task_id TEXT REFERENCES tasks (id) ON DELETE CASCADE,
|
||||
comment_id TEXT REFERENCES comments (id) ON DELETE CASCADE,
|
||||
group_id TEXT REFERENCES project_groups (id) ON DELETE CASCADE,
|
||||
note_id TEXT REFERENCES notes (id) ON DELETE CASCADE,
|
||||
filename TEXT NOT NULL,
|
||||
mime_type TEXT NOT NULL DEFAULT 'application/octet-stream',
|
||||
size INTEGER NOT NULL,
|
||||
path TEXT NOT NULL,
|
||||
uploaded_by TEXT NOT NULL REFERENCES users (id) ON DELETE CASCADE,
|
||||
created_at INTEGER NOT NULL,
|
||||
-- Counted, not spelled out as pairs: with four columns the pairwise version is
|
||||
-- six clauses, and the one that is wrong looks exactly like the one that is right.
|
||||
CHECK ((task_id IS NOT NULL) + (comment_id IS NOT NULL)
|
||||
+ (group_id IS NOT NULL) + (note_id IS NOT NULL) = 1)
|
||||
);
|
||||
|
||||
INSERT INTO attachments_with_notes
|
||||
(id, task_id, comment_id, group_id, filename, mime_type, size, path, uploaded_by, created_at)
|
||||
SELECT id, task_id, comment_id, group_id, filename, mime_type, size, path, uploaded_by, created_at
|
||||
FROM attachments;
|
||||
|
||||
DROP TABLE attachments;
|
||||
|
||||
ALTER TABLE attachments_with_notes RENAME TO attachments;
|
||||
|
||||
CREATE INDEX idx_attachments_task ON attachments (task_id) WHERE task_id IS NOT NULL;
|
||||
CREATE INDEX idx_attachments_comment ON attachments (comment_id) WHERE comment_id IS NOT NULL;
|
||||
CREATE INDEX idx_attachments_group ON attachments (group_id) WHERE group_id IS NOT NULL;
|
||||
CREATE INDEX idx_attachments_note ON attachments (note_id) WHERE note_id IS NOT NULL;
|
||||
@@ -0,0 +1,26 @@
|
||||
-- One row per installation that has ever reported in.
|
||||
--
|
||||
-- This table only exists on the instance that is acting as the collector. Every
|
||||
-- other instance has it too, empty, because the schema is the same everywhere —
|
||||
-- and because "am I the collector" is a setting, not a build.
|
||||
--
|
||||
-- What is deliberately NOT here is the point of the whole feature: there is no
|
||||
-- column for an IP address, a hostname, a domain, an account, a count of tasks,
|
||||
-- or anything else. The two things stored are the two things sent, and the table
|
||||
-- is the promise written down where it cannot drift from the prose — a column
|
||||
-- added here later is a promise broken, and it should be as awkward to add as
|
||||
-- possible.
|
||||
--
|
||||
-- `instance_id` is the primary key rather than a rowid alias, because the same
|
||||
-- installation reporting daily must update its row rather than grow a new one.
|
||||
-- What is wanted is "how many installations", not "how many pings".
|
||||
CREATE TABLE beacon_installs (
|
||||
instance_id TEXT PRIMARY KEY,
|
||||
version TEXT NOT NULL DEFAULT '',
|
||||
first_seen INTEGER NOT NULL,
|
||||
last_seen INTEGER NOT NULL
|
||||
);
|
||||
|
||||
-- The count is always "seen in the last N days", so last_seen is what every query
|
||||
-- sorts and filters on.
|
||||
CREATE INDEX beacon_by_last_seen ON beacon_installs (last_seen DESC);
|
||||
@@ -0,0 +1,111 @@
|
||||
-- Calendars read from somewhere else: the account, the calendars in it, and the
|
||||
-- events they hold.
|
||||
--
|
||||
-- Not in `mailboxes`, and it was the first thing considered. The two look alike
|
||||
-- from a distance — a person connects a Google account over OAuth, tokens are
|
||||
-- sealed, a sweep polls it — and every column past that disagrees. `trigger_kind`,
|
||||
-- `seen`, `last_uid` and `project_id` all answer "what turns a message into a
|
||||
-- task", and nothing here turns into a task. What is here instead is a *set*:
|
||||
-- one account holds many calendars, and a person picks which of them to look at.
|
||||
-- `mailboxes` has no room for a one-to-many, and giving it one would push the
|
||||
-- other kind's columns onto rows that can never use them.
|
||||
--
|
||||
-- The decisive part is smaller and more practical. `mailboxes.kind` carries
|
||||
-- `CHECK (kind IN ('gmail', 'imap'))`, and SQLite cannot alter a CHECK — adding a
|
||||
-- third value means rebuilding the table with foreign keys off, which is the
|
||||
-- expensive, careful thing migration 0021 had to do for a reason. Paying that to
|
||||
-- store something that shares no column with the table is the wrong trade.
|
||||
--
|
||||
-- Hence no CHECK on `provider` here. CalDAV is the next source to read, and it
|
||||
-- should cost a row, not a rebuild.
|
||||
|
||||
-- One connected account per person per provider. The tokens live here rather than
|
||||
-- on each calendar: a refresh token copied onto ten rows is ten copies of one
|
||||
-- secret, ten refreshes racing each other, and ten places to miss when access is
|
||||
-- revoked.
|
||||
--
|
||||
-- `refresh_token` and `access_token` are written sealed; see internal/secret. The
|
||||
-- store's own seal/unseal only covers the settings tables, so this table's methods
|
||||
-- call them by hand — the same fact worth knowing that migration 0014 wrote down.
|
||||
CREATE TABLE calendar_accounts (
|
||||
id TEXT PRIMARY KEY,
|
||||
user_id TEXT NOT NULL REFERENCES users (id) ON DELETE CASCADE,
|
||||
provider TEXT NOT NULL DEFAULT 'google',
|
||||
-- The address the account reports, so the settings page can say which one is
|
||||
-- connected rather than only that one is.
|
||||
account TEXT NOT NULL DEFAULT '',
|
||||
|
||||
refresh_token TEXT NOT NULL DEFAULT '',
|
||||
access_token TEXT NOT NULL DEFAULT '',
|
||||
expires_at INTEGER NOT NULL DEFAULT 0,
|
||||
|
||||
last_sync_at INTEGER NOT NULL DEFAULT 0,
|
||||
created_at INTEGER NOT NULL
|
||||
);
|
||||
|
||||
-- Connecting again replaces what was there, the way Gmail's does: the OAuth flow
|
||||
-- is one account at a time, so a second row for the same provider would be a
|
||||
-- second set of tokens for the same grant and no way to say which is live.
|
||||
CREATE UNIQUE INDEX calendar_accounts_once ON calendar_accounts (user_id, provider);
|
||||
|
||||
-- One row per calendar the account can see, kept whether or not it is shown.
|
||||
--
|
||||
-- `shown` rather than deleting the ones that are not: a person who turns a calendar
|
||||
-- off and on again should get the same row back, and the list is refreshed from
|
||||
-- Google on every sync — so a calendar that only existed while it was ticked would
|
||||
-- lose its choice every time somebody added a new one.
|
||||
--
|
||||
-- `colour` is Google's own hex for the calendar. Not one of verdande's ten named
|
||||
-- colours, which is the rule everywhere else in this database: those are names so
|
||||
-- the interface can restyle them per theme, and they are *verdande's* palette. A
|
||||
-- Google calendar is told apart by the colour the person already knows it by, and
|
||||
-- that colour belongs to Google.
|
||||
CREATE TABLE calendars (
|
||||
id TEXT PRIMARY KEY,
|
||||
account_id TEXT NOT NULL REFERENCES calendar_accounts (id) ON DELETE CASCADE,
|
||||
-- The id Google knows it by. Long, and an address for the primary calendar.
|
||||
remote_id TEXT NOT NULL,
|
||||
name TEXT NOT NULL DEFAULT '',
|
||||
colour TEXT NOT NULL DEFAULT '',
|
||||
time_zone TEXT NOT NULL DEFAULT '',
|
||||
primary_one INTEGER NOT NULL DEFAULT 0,
|
||||
writable INTEGER NOT NULL DEFAULT 0,
|
||||
shown INTEGER NOT NULL DEFAULT 0,
|
||||
sort_order INTEGER NOT NULL DEFAULT 0
|
||||
);
|
||||
|
||||
CREATE UNIQUE INDEX calendars_once ON calendars (account_id, remote_id);
|
||||
|
||||
-- The events themselves, cached.
|
||||
--
|
||||
-- Cached rather than fetched per view, and the reason is what a month grid does:
|
||||
-- it pages. Asking Google on every arrow press means a token refresh, a round trip
|
||||
-- and a spinner for a question that was answered ninety seconds ago — and a
|
||||
-- calendar that is slower than the calendar it is showing is not worth having.
|
||||
-- The window is refreshed by the sweep and by the "fetch now" button.
|
||||
--
|
||||
-- Days rather than instants, and both ends inclusive. A grid asks one question —
|
||||
-- "which cells does this cover" — and `start_day <= ? AND end_day >= ?` is that
|
||||
-- question. `starts_at` is kept as Google wrote it, offset and all, because it is
|
||||
-- what the chip shows and reparsing it here would ask *this process* what time
|
||||
-- zone it is in. An all-day event has neither: it has a day, not a moment, and
|
||||
-- inventing midnight for it is how an event moves a day when a container moves
|
||||
-- time zone.
|
||||
CREATE TABLE calendar_events (
|
||||
id TEXT PRIMARY KEY,
|
||||
calendar_id TEXT NOT NULL REFERENCES calendars (id) ON DELETE CASCADE,
|
||||
remote_id TEXT NOT NULL,
|
||||
summary TEXT NOT NULL DEFAULT '',
|
||||
starts_at TEXT NOT NULL DEFAULT '',
|
||||
ends_at TEXT NOT NULL DEFAULT '',
|
||||
start_day TEXT NOT NULL,
|
||||
end_day TEXT NOT NULL,
|
||||
all_day INTEGER NOT NULL DEFAULT 0,
|
||||
location TEXT NOT NULL DEFAULT '',
|
||||
url TEXT NOT NULL DEFAULT ''
|
||||
);
|
||||
|
||||
-- The one query there is: everything a person can see, between two days. Ordered
|
||||
-- by the account so the join has somewhere to start, then by the days the grid
|
||||
-- compares against.
|
||||
CREATE INDEX calendar_events_by_day ON calendar_events (calendar_id, start_day, end_day);
|
||||
@@ -0,0 +1,23 @@
|
||||
-- A note can be put away without being thrown away.
|
||||
--
|
||||
-- The trash already exists and is the wrong tool. It says "this was a mistake,
|
||||
-- and in thirty days it is gone" — so anything worth keeping cannot go in it, and
|
||||
-- an archive of twelve hundred imported notes is exactly the case where most of
|
||||
-- them are worth keeping and almost none are worth looking at. Without somewhere
|
||||
-- to put them, the list is the whole archive forever, and finding this week's note
|
||||
-- means reading past a decade.
|
||||
--
|
||||
-- A column rather than a folder. Verdande already refused a second hierarchy for
|
||||
-- notes — see 0018 — and archiving is not filing: it is one bit about whether a
|
||||
-- note is current, and it has to be answerable next to `deleted_at`, which is the
|
||||
-- other bit of the same kind.
|
||||
--
|
||||
-- Nullable timestamp rather than a boolean, for the same reason `deleted_at` is
|
||||
-- one: "when was this put away" answers questions a flag cannot, and it costs the
|
||||
-- same byte.
|
||||
ALTER TABLE notes ADD COLUMN archived_at INTEGER;
|
||||
|
||||
-- The list asks "not deleted, not archived, newest first" on every load, and with
|
||||
-- twelve hundred notes that is the query that has to be quick.
|
||||
CREATE INDEX notes_current ON notes (archived_at, deleted_at, updated_at DESC)
|
||||
WHERE deleted_at IS NULL;
|
||||
@@ -0,0 +1,37 @@
|
||||
-- One row per person *and scope*, which is what the table always meant.
|
||||
--
|
||||
-- `user_settings` was created with `user_id` as its whole primary key, and its
|
||||
-- upsert rewrote `scope` on conflict. So a person had exactly one settings row,
|
||||
-- and whichever scope was written last was the only one that survived: saving an
|
||||
-- AI provider deleted the Gmail connection, and connecting a calendar deleted the
|
||||
-- AI provider. Nothing reported it — the write succeeded, and the loss was in the
|
||||
-- row it landed on.
|
||||
--
|
||||
-- This was seen coming twice. Migration 0011 gave the instance's settings their
|
||||
-- own table partly for this reason and wrote the warning down in as many words;
|
||||
-- 0014 moved mailboxes out and said the warning "was nearly walked into a second
|
||||
-- time". The third time was the calendar, which added a `calendar` scope beside
|
||||
-- `ai` and `gmail` — and walked in.
|
||||
--
|
||||
-- The lesson worth keeping: a warning in a migration is read by whoever is editing
|
||||
-- *that* migration, and nobody was. The shape is fixed here so there is nothing
|
||||
-- left to warn about.
|
||||
--
|
||||
-- SQLite cannot change a primary key in place, so the table is rebuilt. Existing
|
||||
-- rows are copied as they are: each person has at most one, and it holds whatever
|
||||
-- scope won last. What the earlier writes lost is already gone and cannot be
|
||||
-- recovered here — the values were overwritten, not orphaned.
|
||||
CREATE TABLE user_settings_new (
|
||||
user_id TEXT NOT NULL REFERENCES users (id) ON DELETE CASCADE,
|
||||
scope TEXT NOT NULL DEFAULT 'general',
|
||||
values_json TEXT NOT NULL DEFAULT '{}',
|
||||
updated_at INTEGER NOT NULL,
|
||||
PRIMARY KEY (user_id, scope)
|
||||
);
|
||||
|
||||
INSERT INTO user_settings_new (user_id, scope, values_json, updated_at)
|
||||
SELECT user_id, scope, values_json, updated_at FROM user_settings;
|
||||
|
||||
DROP TABLE user_settings;
|
||||
|
||||
ALTER TABLE user_settings_new RENAME TO user_settings;
|
||||
@@ -0,0 +1,30 @@
|
||||
-- Abonnement på en kalender, man kun har en adresse til.
|
||||
--
|
||||
-- Google-vejen er OAuth: én konto pr. person, hvor tokenet er identiteten, og
|
||||
-- `calendar_accounts_once` sørgede for netop det — én række pr. (person,
|
||||
-- udbyder). Et abonnement er en anden slags ting. Der er intet token og ingen
|
||||
-- konto, kun en adresse, og man har flere: helligdage, en fodboldklub, en
|
||||
-- fælleskalender fra arbejdet. Det er derfor det blev bedt om som "flere
|
||||
-- kalendere".
|
||||
--
|
||||
-- Samme tabel frem for en ny. Et abonnement er én kalender med begivenheder i, og
|
||||
-- det er præcis det, `calendars` og `calendar_events` allerede beskriver — med
|
||||
-- vinduet, `shown`, farven og hele fejemaskinen omkring sig. En tabel til ville
|
||||
-- være en anden vej til det samme sted, og to veje til ét sted er to steder at
|
||||
-- rette den dag noget om begivenheder ændrer sig.
|
||||
--
|
||||
-- Adressen bliver identiteten, som tokenet er det for Google. To rækker med den
|
||||
-- samme adresse for den samme person ville være den samme kalender hentet to
|
||||
-- gange, og der er ingen måde at sige hvilken der gælder.
|
||||
ALTER TABLE calendar_accounts ADD COLUMN url TEXT NOT NULL DEFAULT '';
|
||||
|
||||
-- Det gamle indeks holder stadig for OAuth-rækkerne — dem uden adresse — og
|
||||
-- slipper abonnementerne fri. Delvise indekser er den ene ting SQLite kan her,
|
||||
-- som en tabelombygning ellers skulle til for.
|
||||
DROP INDEX calendar_accounts_once;
|
||||
|
||||
CREATE UNIQUE INDEX calendar_accounts_once
|
||||
ON calendar_accounts (user_id, provider) WHERE url = '';
|
||||
|
||||
CREATE UNIQUE INDEX calendar_subscription_once
|
||||
ON calendar_accounts (user_id, url) WHERE url <> '';
|
||||
@@ -10,15 +10,26 @@ import (
|
||||
|
||||
// Note is a page of Markdown belonging to a project, or to nobody.
|
||||
type Note struct {
|
||||
ID string `json:"id"`
|
||||
ProjectID string `json:"project_id,omitempty"`
|
||||
Title string `json:"title"`
|
||||
Body string `json:"body"`
|
||||
CreatedBy string `json:"created_by,omitempty"`
|
||||
Pinned bool `json:"pinned"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
DeletedAt *time.Time `json:"deleted_at,omitempty"`
|
||||
ID string `json:"id"`
|
||||
ProjectID string `json:"project_id,omitempty"`
|
||||
Title string `json:"title"`
|
||||
Body string `json:"body"`
|
||||
CreatedBy string `json:"created_by,omitempty"`
|
||||
Pinned bool `json:"pinned"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
DeletedAt *time.Time `json:"deleted_at,omitempty"`
|
||||
ArchivedAt *time.Time `json:"archived_at,omitempty"`
|
||||
|
||||
// Preview is the first of the body, for a list that shows one line of it.
|
||||
//
|
||||
// It exists so a list of twelve hundred notes does not have to carry twelve
|
||||
// hundred whole notes: the list rendered 845 KB of JSON to show ninety
|
||||
// characters of each. When it is set, Body is deliberately empty — see
|
||||
// handleListNotes. That is not tidiness but a guard: an editor handed a
|
||||
// truncated body would save the truncation, and the note would be cut down to
|
||||
// its own preview.
|
||||
Preview string `json:"preview,omitempty"`
|
||||
|
||||
// Links is what the body points at, read out of the text on the way in. Never
|
||||
// set by a caller: it is derived, and a caller that set it would be describing
|
||||
@@ -33,7 +44,16 @@ type NoteLink struct {
|
||||
}
|
||||
|
||||
const noteColumns = `id, project_id, title, body, created_by, pinned,
|
||||
created_at, updated_at, deleted_at`
|
||||
created_at, updated_at, deleted_at, archived_at`
|
||||
|
||||
// The same columns, said with the table in front of them.
|
||||
//
|
||||
// Needed only by the search, which joins `notes_fts` — and that table carries its
|
||||
// own `id` and `project_id` (UNINDEXED, so the row can be found from a match).
|
||||
// Unqualified, SQLite cannot tell which one is meant and refuses the query.
|
||||
const noteColumnsQualified = `notes.id, notes.project_id, notes.title, notes.body,
|
||||
notes.created_by, notes.pinned, notes.created_at, notes.updated_at, notes.deleted_at,
|
||||
notes.archived_at`
|
||||
|
||||
// Reference syntax, and deliberately the same characters the quick-add parser
|
||||
// already uses. A `#Firma` in a note means the same project as a `#Firma` typed
|
||||
@@ -62,11 +82,12 @@ func (db *DB) SaveNote(ctx context.Context, n *Note) error {
|
||||
}
|
||||
n.UpdatedAt = now
|
||||
|
||||
// A note with no title takes its first line, the way Apple Notes does. A list
|
||||
// of untitled notes is a list nobody can read.
|
||||
if strings.TrimSpace(n.Title) == "" {
|
||||
n.Title = firstLine(n.Body)
|
||||
}
|
||||
// The title is always the first line, the way Apple Notes does it. Not "when it
|
||||
// is empty": derived once and then left alone, it goes stale the moment somebody
|
||||
// rewrites the opening of a note, and the list ends up calling a note by a name
|
||||
// that is nowhere in it. The column is a cache of the body, not a field beside
|
||||
// it, and every caller gets the same answer because none of them can set it.
|
||||
n.Title = firstLine(n.Body)
|
||||
|
||||
return db.Tx(ctx, func(tx *sql.Tx) error {
|
||||
var projectID any
|
||||
@@ -127,7 +148,14 @@ func LinksIn(body string) []NoteLink {
|
||||
add("task", strings.ToLower(m[1]))
|
||||
}
|
||||
for _, m := range projectRef.FindAllStringSubmatch(body, -1) {
|
||||
add("project", m[1])
|
||||
// Folded, because #garageristeriet and #GarageRisteriet are the same project
|
||||
// to everybody except a database. The key is stored folded and looked up
|
||||
// folded, so the comparison stays an exact match and keeps using its index —
|
||||
// a comparison that lowered the column instead would read the whole table.
|
||||
//
|
||||
// The project's real spelling is not lost: it is on the project, which is
|
||||
// what the interface shows. This column is a key, not a label.
|
||||
add("project", strings.ToLower(m[1]))
|
||||
}
|
||||
for _, m := range noteRef.FindAllStringSubmatch(body, -1) {
|
||||
add("note", strings.TrimSpace(m[1]))
|
||||
@@ -176,11 +204,57 @@ func (db *DB) NotesInProject(ctx context.Context, projectID string) ([]Note, err
|
||||
projectID)
|
||||
}
|
||||
|
||||
// LooseNotes are the ones filed under no project — a person's own, the way a task
|
||||
// in the inbox is. Nobody else can see them, so there is no role to ask about.
|
||||
func (db *DB) LooseNotes(ctx context.Context, userID string) ([]Note, error) {
|
||||
return db.notesWhere(ctx, `
|
||||
WHERE project_id IS NULL AND created_by = ? AND deleted_at IS NULL
|
||||
ORDER BY pinned DESC, updated_at DESC`, userID)
|
||||
}
|
||||
|
||||
// AllNotes is everything a person can see: their own loose ones, and the ones in
|
||||
// projects they are part of.
|
||||
//
|
||||
// This is what the Notes page lists. Filing a note in a project is how it gets
|
||||
// shared, and a list that dropped it at that moment would make sharing look like
|
||||
// deleting — which is exactly how it read before this existed.
|
||||
func (db *DB) AllNotes(ctx context.Context, userID string) ([]Note, error) {
|
||||
return db.allNotes(ctx, userID, false)
|
||||
}
|
||||
|
||||
// ArchivedNotes is the same list, the other way round: what has been put away.
|
||||
func (db *DB) ArchivedNotes(ctx context.Context, userID string) ([]Note, error) {
|
||||
return db.allNotes(ctx, userID, true)
|
||||
}
|
||||
|
||||
func (db *DB) allNotes(ctx context.Context, userID string, archived bool) ([]Note, error) {
|
||||
// Archived notes are left out of the ordinary list rather than mixed in and
|
||||
// greyed: the point of putting something away is not seeing it.
|
||||
state := "notes.archived_at IS NULL"
|
||||
if archived {
|
||||
state = "notes.archived_at IS NOT NULL"
|
||||
}
|
||||
return db.notesJoin(ctx, noteColumnsQualified, `
|
||||
WHERE `+state+` AND deleted_at IS NULL
|
||||
AND (
|
||||
(project_id IS NULL AND created_by = ?)
|
||||
OR project_id IN (
|
||||
SELECT id FROM projects WHERE owner_id = ? AND deleted_at IS NULL
|
||||
UNION
|
||||
SELECT project_id FROM project_members WHERE user_id = ?
|
||||
)
|
||||
)
|
||||
ORDER BY pinned DESC, updated_at DESC`, userID, userID, userID)
|
||||
}
|
||||
|
||||
// NotesLinking answers the backwards question: what points at this thing.
|
||||
//
|
||||
// The reason note_links exists. Without it this would mean reading every note's
|
||||
// prose, and a panel that slow is one nobody leaves open.
|
||||
func (db *DB) NotesLinking(ctx context.Context, kind, targetID string) ([]Note, error) {
|
||||
if kind == "project" {
|
||||
targetID = strings.ToLower(targetID)
|
||||
}
|
||||
return db.notesWhere(ctx, `
|
||||
WHERE deleted_at IS NULL
|
||||
AND id IN (SELECT note_id FROM note_links WHERE kind = ? AND target_id = ?)
|
||||
@@ -199,13 +273,141 @@ func (db *DB) SearchNotes(ctx context.Context, query string, limit int) ([]Note,
|
||||
if expr == "" {
|
||||
return nil, nil
|
||||
}
|
||||
// By rowid, the way tasks do it: `id` is UNINDEXED in the table, so selecting
|
||||
// on it would work and read the whole index to do it.
|
||||
return db.notesWhere(ctx, `
|
||||
WHERE deleted_at IS NULL
|
||||
AND rowid IN (SELECT rowid FROM notes_fts WHERE notes_fts MATCH ?)
|
||||
ORDER BY updated_at DESC
|
||||
// Ordered by how well it matched, not by when it was last touched.
|
||||
//
|
||||
// `ORDER BY updated_at DESC` was invisible while there were ten notes and
|
||||
// useless at twelve hundred: every one of them was imported the same evening,
|
||||
// so `updated_at` says nothing, and the limit then cut the list at fifty
|
||||
// arbitrary rows. Searching "Tesla" answered with a note about nginx first and
|
||||
// the note actually called "Tesla Model Y" second — which reads as a search
|
||||
// that does not work, because from where the person is sitting it is one.
|
||||
//
|
||||
// bm25 with the title weighted ten times the body. A word in the title is what
|
||||
// the note is *about*; the same word once in a long body is a mention. `fold`
|
||||
// carries the same text transliterated, so it must weigh the same as the body
|
||||
// or a Danish match would outrank an exact one.
|
||||
//
|
||||
// Joined rather than `rowid IN (…)`: the rank belongs to the match, and a
|
||||
// subquery throws it away.
|
||||
found, err := db.notesJoin(ctx, noteColumnsQualified, `
|
||||
JOIN notes_fts ON notes_fts.rowid = notes.rowid
|
||||
WHERE notes.deleted_at IS NULL
|
||||
AND notes_fts MATCH ?
|
||||
ORDER BY bm25(notes_fts, 10.0, 1.0, 1.0)
|
||||
LIMIT ?`, expr, limit)
|
||||
if err != nil || len(found) > 0 {
|
||||
return found, err
|
||||
}
|
||||
return db.searchNotesInside(ctx, query, limit)
|
||||
}
|
||||
|
||||
// searchNotesInside finds a word that lives *inside* another word.
|
||||
//
|
||||
// An index is built of tokens, and a token is what a tokenizer decided was a
|
||||
// word. `gamerule keepInventory true` is three of them, and the middle one is
|
||||
// `keepinventory` — so "keep inventory" asks for two words that are nowhere, and
|
||||
// gets nothing. Reported as "søgning har ikke det store held", which from where
|
||||
// the person is sitting is exactly what it is: the note is right there, and the
|
||||
// search says there is nothing.
|
||||
//
|
||||
// A prefix query gets half of it. `keep*` does match `keepinventory`; `inventory*`
|
||||
// cannot, because it is not a prefix of anything — and the two are joined with AND,
|
||||
// so the half that works is thrown away by the half that cannot.
|
||||
//
|
||||
// This is not an exotic case in a notes app. A decade of notes is full of
|
||||
// commands, identifiers and file names, and every one of them is several words the
|
||||
// index thinks is one: `keepInventory`, `VACUUM INTO`, `apple-notes-til-markdown`.
|
||||
//
|
||||
// So: the index answers first, and when it has nothing, the text is read. Every
|
||||
// term has to appear somewhere in the note, anywhere, in any word.
|
||||
//
|
||||
// # What it costs, and when to stop doing it this way
|
||||
//
|
||||
// A LIKE with a leading % cannot use an index; this reads every note. At twelve
|
||||
// hundred notes that is four megabytes and a millisecond or two, and it only runs
|
||||
// when the fast path already came back empty — which is the search that was about
|
||||
// to disappoint somebody. At a hundred thousand notes it would be the wrong shape,
|
||||
// and the answer then is an FTS5 `trigram` index, not a bigger scan.
|
||||
//
|
||||
// It does not rank. bm25 needs a match to rank, and there is no match here — that
|
||||
// is the whole situation. The rows are the most recently touched first, and the
|
||||
// set is small, because the index already said there was nothing in it.
|
||||
func (db *DB) searchNotesInside(ctx context.Context, query string, limit int) ([]Note, error) {
|
||||
terms := SearchTerms(query)
|
||||
if len(terms) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
// `fold` is `title || ' ' || body` with the Danish letters transliterated, so
|
||||
// one column covers the title, the body and "gron" for "grøn" at once. LIKE is
|
||||
// case-insensitive for ASCII in SQLite, which is what makes `keepInventory`
|
||||
// answer to "inventory".
|
||||
//
|
||||
// The needle is safe to interpolate into a pattern because `tokenize` splits on
|
||||
// everything that is not a letter or a digit: a term cannot contain % or _, so
|
||||
// there is no wildcard for somebody to smuggle in. It is still a bound
|
||||
// parameter — that is about SQL, and this is about LIKE.
|
||||
where := "WHERE deleted_at IS NULL"
|
||||
args := make([]any, 0, len(terms)+1)
|
||||
for _, term := range terms {
|
||||
where += " AND fold LIKE ?"
|
||||
args = append(args, "%"+FoldDanish(term)+"%")
|
||||
}
|
||||
args = append(args, limit)
|
||||
|
||||
return db.notesWhere(ctx, where+" ORDER BY updated_at DESC LIMIT ?", args...)
|
||||
}
|
||||
|
||||
// SetNoteTimes writes the dates a note had somewhere else.
|
||||
//
|
||||
// Only the import calls this, and it exists because SaveNote is right to refuse:
|
||||
// `updated_at` is when this program last wrote the note, and letting any caller
|
||||
// set it would make the column mean whatever the last writer felt like. A note
|
||||
// brought in from Apple Notes is the one case where the dates are facts from
|
||||
// before this database existed — and without them a move of twelve hundred notes
|
||||
// is a pile that was all written the same evening, which throws away the order
|
||||
// they were written in. That order is half of what an archive is.
|
||||
func (db *DB) SetNoteTimes(ctx context.Context, id string, created, updated time.Time) error {
|
||||
if created.IsZero() && updated.IsZero() {
|
||||
return nil
|
||||
}
|
||||
// COALESCE, so a note carrying only one of the two keeps what it already had
|
||||
// for the other rather than being stamped with the zero time — which would sort
|
||||
// it to the year one.
|
||||
var c, u any
|
||||
if !created.IsZero() {
|
||||
c = created.Unix()
|
||||
}
|
||||
if !updated.IsZero() {
|
||||
u = updated.Unix()
|
||||
}
|
||||
_, err := db.ExecContext(ctx, `
|
||||
UPDATE notes
|
||||
SET created_at = COALESCE(?, created_at),
|
||||
updated_at = COALESCE(?, updated_at)
|
||||
WHERE id = ?`, c, u, id)
|
||||
return err
|
||||
}
|
||||
|
||||
// SetNoteArchived puts a note away, or takes it back out.
|
||||
//
|
||||
// Separate from DeleteNote because they mean different things and the difference
|
||||
// is the whole feature: the trash says "this was a mistake, and in thirty days it
|
||||
// is gone", and the archive says "this is finished, and I want to keep it". A
|
||||
// twelve-hundred-note import is mostly the second, and without somewhere to put
|
||||
// them the list is the whole archive forever.
|
||||
//
|
||||
// `updated_at` is deliberately not touched. Archiving is not editing, and a list
|
||||
// sorted by when things were last written should not be reordered by somebody
|
||||
// tidying up.
|
||||
func (db *DB) SetNoteArchived(ctx context.Context, id string, archived bool) error {
|
||||
var at any
|
||||
if archived {
|
||||
at = time.Now().Unix()
|
||||
}
|
||||
_, err := db.ExecContext(ctx,
|
||||
`UPDATE notes SET archived_at = ? WHERE id = ? AND deleted_at IS NULL`, at, id)
|
||||
return err
|
||||
}
|
||||
|
||||
// DeleteNote puts it in the trash. Nothing here removes a row.
|
||||
@@ -225,7 +427,13 @@ func (db *DB) RestoreNote(ctx context.Context, id string) error {
|
||||
}
|
||||
|
||||
func (db *DB) notesWhere(ctx context.Context, where string, args ...any) ([]Note, error) {
|
||||
rows, err := db.QueryContext(ctx, `SELECT `+noteColumns+` FROM notes `+where, args...)
|
||||
return db.notesJoin(ctx, noteColumns, where, args...)
|
||||
}
|
||||
|
||||
// notesJoin is notesWhere with the column list spelled out, for the one caller
|
||||
// that joins another table and must therefore say which `id` it means.
|
||||
func (db *DB) notesJoin(ctx context.Context, columns, where string, args ...any) ([]Note, error) {
|
||||
rows, err := db.QueryContext(ctx, `SELECT `+columns+` FROM notes `+where, args...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -265,10 +473,10 @@ func scanNote(row scanner) (Note, error) {
|
||||
var n Note
|
||||
var projectID, createdBy sql.NullString
|
||||
var created, updated int64
|
||||
var deleted sql.NullInt64
|
||||
var deleted, archived sql.NullInt64
|
||||
|
||||
if err := row.Scan(&n.ID, &projectID, &n.Title, &n.Body, &createdBy,
|
||||
&n.Pinned, &created, &updated, &deleted); err != nil {
|
||||
&n.Pinned, &created, &updated, &deleted, &archived); err != nil {
|
||||
return n, err
|
||||
}
|
||||
n.ProjectID = projectID.String
|
||||
@@ -279,5 +487,9 @@ func scanNote(row scanner) (Note, error) {
|
||||
t := time.Unix(deleted.Int64, 0)
|
||||
n.DeletedAt = &t
|
||||
}
|
||||
if archived.Valid {
|
||||
t := time.Unix(archived.Int64, 0)
|
||||
n.ArchivedAt = &t
|
||||
}
|
||||
return n, nil
|
||||
}
|
||||
|
||||
@@ -29,7 +29,10 @@ func TestATagInANoteIsTheSameTagAsInATask(t *testing.T) {
|
||||
if len(projects) != 2 {
|
||||
t.Fatalf("read %v out of the note, want the two projects", n.Links)
|
||||
}
|
||||
if projects[0] != "Firma" || projects[1] != "Regnskab" {
|
||||
// Folded: the key is a key, not a label. #firma and #Firma are the same project
|
||||
// to everybody except a database, and the project's own spelling is on the
|
||||
// project, which is what the interface shows.
|
||||
if projects[0] != "firma" || projects[1] != "regnskab" {
|
||||
t.Errorf("got %v", projects)
|
||||
}
|
||||
}
|
||||
@@ -118,10 +121,10 @@ func TestNotesAreFoundInEitherSpelling(t *testing.T) {
|
||||
db, userID := sealedStore(t)
|
||||
ctx := context.Background()
|
||||
|
||||
// The title is the first line now, so that is where it goes.
|
||||
if err := db.SaveNote(ctx, &Note{
|
||||
CreatedBy: userID,
|
||||
Title: "Grønt regnskab",
|
||||
Body: "Målt på årsbasis for Århus.",
|
||||
Body: "Grønt regnskab\n\nMålt på årsbasis for Århus.",
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -173,7 +176,9 @@ func TestWhatIsReadOutOfABody(t *testing.T) {
|
||||
{"a hash inside a word is not a tag", "C#Sharp og x#y", nil},
|
||||
{"a note by title", "se [[Møde med Anders]]", []NoteLink{{"note", "Møde med Anders"}}},
|
||||
{"the same tag twice is one link", "#Firma og #Firma igen",
|
||||
[]NoteLink{{"project", "Firma"}}},
|
||||
[]NoteLink{{"project", "firma"}}},
|
||||
{"two spellings are one tag", "#Firma og #firma",
|
||||
[]NoteLink{{"project", "firma"}}},
|
||||
{"a heading is not a tag", "# Overskrift", nil},
|
||||
} {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
@@ -245,3 +250,186 @@ func TestTheProjectCountIsWhatIsLeft(t *testing.T) {
|
||||
t.Errorf("after deleting one, the count is %d, want 1", n)
|
||||
}
|
||||
}
|
||||
|
||||
// The title follows the first line, always — not only the first time.
|
||||
//
|
||||
// Derived once and then left alone, it goes stale the moment somebody rewrites
|
||||
// the opening, and the list ends up calling a note by a name that is nowhere in
|
||||
// it. That is exactly what happened: a note headed "Ny note" was listed as
|
||||
// "fdgfgfgh", which was what its first line had been some edits ago.
|
||||
func TestTheTitleFollowsTheFirstLine(t *testing.T) {
|
||||
db, userID := sealedStore(t)
|
||||
ctx := context.Background()
|
||||
|
||||
n := &Note{CreatedBy: userID, Body: "Første udgave\n\nnoget tekst"}
|
||||
if err := db.SaveNote(ctx, n); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if n.Title != "Første udgave" {
|
||||
t.Fatalf("title started as %q", n.Title)
|
||||
}
|
||||
|
||||
n.Body = "# Ny note\n\nDette er mit nye noteprogram"
|
||||
if err := db.SaveNote(ctx, n); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if n.Title != "Ny note" {
|
||||
t.Errorf("after rewriting the first line the title is %q", n.Title)
|
||||
}
|
||||
|
||||
back, err := db.Note(ctx, n.ID)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if back.Title != "Ny note" {
|
||||
t.Errorf("the stored title is %q", back.Title)
|
||||
}
|
||||
|
||||
// And a title cannot be set beside the text, because then the two can disagree.
|
||||
n.Title = "noget andet"
|
||||
if err := db.SaveNote(ctx, n); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if n.Title != "Ny note" {
|
||||
t.Errorf("a title set by hand survived as %q", n.Title)
|
||||
}
|
||||
}
|
||||
|
||||
// A note that names a project in the other case must still turn up on it.
|
||||
//
|
||||
// It did not, and nothing said why: the key was stored exactly as typed and
|
||||
// looked up the same way, so #garageristeriet and #GarageRisteriet were two
|
||||
// different things that looked identical to a person.
|
||||
func TestATagFindsItsProjectWhateverTheCase(t *testing.T) {
|
||||
db, userID := sealedStore(t)
|
||||
ctx := context.Background()
|
||||
|
||||
if err := db.SaveNote(ctx, &Note{
|
||||
CreatedBy: userID,
|
||||
Body: "Bønnerne kommer fredag #garageristeriet",
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
for _, spelling := range []string{"GarageRisteriet", "garageristeriet", "GARAGERISTERIET"} {
|
||||
found, err := db.NotesLinking(ctx, "project", spelling)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(found) != 1 {
|
||||
t.Errorf("looking up %q found %d notes, want 1", spelling, len(found))
|
||||
}
|
||||
}
|
||||
|
||||
// A note title is a title, not a key: two notes called "Møde" and "møde" are
|
||||
// two notes, and folding them together would merge things somebody kept apart.
|
||||
if err := db.SaveNote(ctx, &Note{CreatedBy: userID, Body: "se [[Møde]]"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if found, _ := db.NotesLinking(ctx, "note", "møde"); len(found) != 0 {
|
||||
t.Error("a note title was folded")
|
||||
}
|
||||
if found, _ := db.NotesLinking(ctx, "note", "Møde"); len(found) != 1 {
|
||||
t.Error("a note title did not match itself")
|
||||
}
|
||||
}
|
||||
|
||||
// En søgning skal svare med det, der passer bedst, øverst.
|
||||
//
|
||||
// Den sorterede efter `updated_at`, hvilket var usynligt med ti noter og
|
||||
// ubrugeligt med tolv hundrede: de blev alle importeret samme aften, så datoen
|
||||
// siger ingenting, og loftet skar så listen af ved halvtreds tilfældige rækker.
|
||||
// At søge "Tesla" svarede med en note om nginx først og noten, der hedder "Tesla
|
||||
// Model Y", bagefter — hvilket læses som en søgning, der ikke virker, fordi det
|
||||
// fra brugerens plads er præcis dét.
|
||||
func TestSearchAnswersWithTheBestMatchFirst(t *testing.T) {
|
||||
db, userID := sealedStore(t)
|
||||
ctx := context.Background()
|
||||
|
||||
// Skrevet i den rækkefølge, der ville give det forkerte svar: den, der kun
|
||||
// nævner ordet i kroppen, gemmes sidst og har derfor den nyeste dato.
|
||||
for _, body := range []string{
|
||||
"# Tesla Model Y\n\nBilen selv.\n",
|
||||
"# Ladestandere\n\nNoter om Tesla og andre mærker.\n",
|
||||
"# nginx proxy manager backup\n\nEn konfiguration. Tesla nævnes i forbifarten.\n",
|
||||
} {
|
||||
if err := db.SaveNote(ctx, &Note{CreatedBy: userID, Body: body}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
found, err := db.SearchNotes(ctx, "Tesla", 10)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(found) != 3 {
|
||||
t.Fatalf("fandt %d noter, ventede 3", len(found))
|
||||
}
|
||||
if found[0].Title != "Tesla Model Y" {
|
||||
t.Errorf("øverst stod %q; den note, der hedder det, man søgte efter, skal stå først", found[0].Title)
|
||||
}
|
||||
}
|
||||
|
||||
// A word inside a word.
|
||||
//
|
||||
// Reported as "søgning har ikke det store held": a note holding
|
||||
// `gamerule keepInventory true` could not be found by typing "keep inventory".
|
||||
// The index is built of tokens, and `keepInventory` is one token — so the two
|
||||
// words are nowhere, and the search says there is nothing while the note is right
|
||||
// there.
|
||||
//
|
||||
// A decade of notes is full of these: commands, identifiers, file names. Every one
|
||||
// of them is several words the index thinks is one.
|
||||
func TestAWordInsideAWordIsStillFound(t *testing.T) {
|
||||
db, userID := sealedStore(t)
|
||||
ctx := context.Background()
|
||||
|
||||
if err := db.SaveNote(ctx, &Note{
|
||||
CreatedBy: userID,
|
||||
Body: "Minecraft Bedrock\n\ngamerule keepInventory true",
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// A second note, so a fallback that simply returns everything would be caught.
|
||||
if err := db.SaveNote(ctx, &Note{
|
||||
CreatedBy: userID,
|
||||
Body: "Indkøb\n\nKaffe og filtre",
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
for _, q := range []string{
|
||||
"keep inventory", // the two words, which is how somebody looks for it
|
||||
"inventory", // the tail on its own — no prefix query can reach this
|
||||
"keepinventory", // written as one, which the index does hold
|
||||
"KEEP INVENTORY", // and case is not a question anybody is asking
|
||||
"vacuum", // not in either note
|
||||
} {
|
||||
found, err := db.SearchNotes(ctx, q, 10)
|
||||
if err != nil {
|
||||
t.Fatalf("%q: %v", q, err)
|
||||
}
|
||||
want := 1
|
||||
if q == "vacuum" {
|
||||
want = 0
|
||||
}
|
||||
if len(found) != want {
|
||||
t.Errorf("%q found %d notes, want %d", q, len(found), want)
|
||||
continue
|
||||
}
|
||||
if want == 1 && found[0].Title != "Minecraft Bedrock" {
|
||||
t.Errorf("%q found %q", q, found[0].Title)
|
||||
}
|
||||
}
|
||||
|
||||
// The fast path is still the fast path: a word the index does hold must not be
|
||||
// answered by the scan, or the ranking that was built for twelve hundred notes
|
||||
// is quietly gone.
|
||||
found, err := db.SearchNotes(ctx, "Bedrock", 10)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(found) != 1 {
|
||||
t.Fatalf("an ordinary word found %d notes", len(found))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -140,11 +140,17 @@ func (db *DB) SavePushSubscription(ctx context.Context, sub *PushSubscription) e
|
||||
if sub.ID == "" {
|
||||
sub.ID = NewID()
|
||||
}
|
||||
// The upsert no longer moves the row between accounts.
|
||||
//
|
||||
// `user_id = excluded.user_id` was in the DO UPDATE, so anybody who learned
|
||||
// another person's endpoint could claim it by subscribing with it: the
|
||||
// victim's device would then receive the claimant's notifications and none of
|
||||
// its own. A re-subscribe from the same person still refreshes the keys, which
|
||||
// is what the upsert is actually for.
|
||||
_, err := db.ExecContext(ctx,
|
||||
`INSERT INTO push_subscriptions (id, user_id, endpoint, p256dh, auth, user_agent, created_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT (endpoint) DO UPDATE SET
|
||||
user_id = excluded.user_id,
|
||||
p256dh = excluded.p256dh,
|
||||
auth = excluded.auth,
|
||||
user_agent = excluded.user_agent,
|
||||
@@ -175,7 +181,21 @@ func (db *DB) ListPushSubscriptions(ctx context.Context, userID string) ([]PushS
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
func (db *DB) DeletePushSubscription(ctx context.Context, endpoint string) error {
|
||||
// DeletePushSubscription removes one of this person's devices.
|
||||
//
|
||||
// Scoped by user as well as by endpoint. It was endpoint alone, which meant
|
||||
// anybody who learned somebody else's endpoint could silently unsubscribe their
|
||||
// phone — and the endpoint is the only thing about a subscription that ever
|
||||
// leaves the browser.
|
||||
func (db *DB) DeletePushSubscription(ctx context.Context, userID, endpoint string) error {
|
||||
_, err := db.ExecContext(ctx,
|
||||
`DELETE FROM push_subscriptions WHERE endpoint = ? AND user_id = ?`, endpoint, userID)
|
||||
return err
|
||||
}
|
||||
|
||||
// dropPushEndpoint removes a subscription the push service has declared dead.
|
||||
// Unexported so the only *request* path to a delete stays the scoped one.
|
||||
func (db *DB) dropPushEndpoint(ctx context.Context, endpoint string) error {
|
||||
_, err := db.ExecContext(ctx, `DELETE FROM push_subscriptions WHERE endpoint = ?`, endpoint)
|
||||
return err
|
||||
}
|
||||
@@ -186,7 +206,10 @@ func (db *DB) DeletePushSubscription(ctx context.Context, endpoint string) error
|
||||
// silently unsubscribing somebody's phone.
|
||||
func (db *DB) RecordPushFailure(ctx context.Context, endpoint string, permanent bool) error {
|
||||
if permanent {
|
||||
return db.DeletePushSubscription(ctx, endpoint)
|
||||
// Not scoped by user, and that is right here: this is the push service
|
||||
// saying the endpoint is gone, not a person asking for something. There is
|
||||
// no caller to check against — the caller is the delivery job.
|
||||
return db.dropPushEndpoint(ctx, endpoint)
|
||||
}
|
||||
_, err := db.ExecContext(ctx,
|
||||
`UPDATE push_subscriptions SET failures = failures + 1 WHERE endpoint = ?`, endpoint)
|
||||
@@ -289,6 +312,11 @@ var secretFields = map[string]bool{
|
||||
"access_token": true,
|
||||
"password": true,
|
||||
"client_secret": true,
|
||||
// The AI provider's key. Missed when this list was written, and it is exactly
|
||||
// the same kind of thing as the rest: a live credential, in a settings blob,
|
||||
// in a database anybody with a backup file can read. Existing rows convert
|
||||
// themselves the next time the settings are saved.
|
||||
"api_key": true,
|
||||
}
|
||||
|
||||
// seal returns a copy of values with the secret fields sealed. Without a key it
|
||||
@@ -363,11 +391,15 @@ func (db *DB) SetUserSettings(ctx context.Context, userID, scope string, values
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
// On (user_id, scope), which is the key since 0025. It used to be on `user_id`
|
||||
// alone and set `scope = excluded.scope` — so writing one scope did not add a
|
||||
// row beside the others, it took theirs and renamed it. The `ai`, `gmail` and
|
||||
// `calendar` settings each deleted the other two, and the write reported
|
||||
// success every time.
|
||||
_, err = db.ExecContext(ctx, `
|
||||
INSERT INTO user_settings (user_id, scope, values_json, updated_at)
|
||||
VALUES (?, ?, ?, ?)
|
||||
ON CONFLICT (user_id) DO UPDATE SET
|
||||
scope = excluded.scope,
|
||||
ON CONFLICT (user_id, scope) DO UPDATE SET
|
||||
values_json = excluded.values_json,
|
||||
updated_at = excluded.updated_at`,
|
||||
userID, scope, string(raw), time.Now().Unix())
|
||||
|
||||
@@ -646,6 +646,35 @@ func (db *DB) ReorderProjects(ctx context.Context, userID string, ids []string)
|
||||
})
|
||||
}
|
||||
|
||||
// ReorderSections writes the order of one project's sections, top to bottom.
|
||||
//
|
||||
// The whole list at once, the same as ReorderProjects and for the same reasons: a
|
||||
// project has a handful of sections, so writing 0, 1, 2 … is one small
|
||||
// transaction that cannot run out of precision, and a list is impossible to get
|
||||
// into the half-ordered state a sequence of individual moves leaves behind when
|
||||
// one of them fails.
|
||||
//
|
||||
// Scoped to the project rather than to a user. A section has no owner of its own —
|
||||
// it belongs to a project, and whether the caller may rearrange it is a question
|
||||
// about that project, which the handler has already asked. Ids belonging to some
|
||||
// other project are skipped by the WHERE clause rather than refused, so a stale
|
||||
// list from a client that has just moved a section elsewhere cannot rearrange a
|
||||
// project the caller was never looking at.
|
||||
func (db *DB) ReorderSections(ctx context.Context, projectID string, ids []string) error {
|
||||
return db.Tx(ctx, func(tx *sql.Tx) error {
|
||||
now := time.Now().Unix()
|
||||
for i, id := range ids {
|
||||
if _, err := tx.ExecContext(ctx,
|
||||
`UPDATE sections SET sort_order = ?, updated_at = ?
|
||||
WHERE id = ? AND project_id = ? AND deleted_at IS NULL`,
|
||||
float64(i), now, id, projectID); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
// SectionByName resolves a "/name" from quick add within one project.
|
||||
//
|
||||
// Case-insensitive, and scoped to the project the task is landing in — a section
|
||||
|
||||
@@ -20,6 +20,13 @@ func FoldDanish(s string) string {
|
||||
return r.Replace(s)
|
||||
}
|
||||
|
||||
// The ceilings on a search. See MatchExprOver for why they exist; both are far
|
||||
// above anything a person types and far below anything that costs.
|
||||
const (
|
||||
maxSearchInput = 512
|
||||
maxSearchTerms = 16
|
||||
)
|
||||
|
||||
// MatchExpr turns what someone typed into the search box into an FTS5 MATCH
|
||||
// expression.
|
||||
//
|
||||
@@ -51,6 +58,23 @@ func MatchExprOver(query string, columns ...string) string {
|
||||
return ""
|
||||
}
|
||||
|
||||
// Capped, because the expression is built from the input and then run inside
|
||||
// SQLite on one of a handful of pooled connections.
|
||||
//
|
||||
// Every term becomes roughly fifty bytes of FTS5, and every one of them is a
|
||||
// prefix query — which is what makes ⌘K feel like it searches as you type, and
|
||||
// also what makes each one expensive. A quarter of a megabyte pasted into the
|
||||
// box became a two-megabyte MATCH expression and half a minute inside SQLite,
|
||||
// holding a connection the whole time. A few of those at once and the database
|
||||
// is unavailable to everybody on the instance, logins included.
|
||||
//
|
||||
// Sixteen terms is far past anything a person types into a search box and
|
||||
// nowhere near enough to hurt. Silently, because a message about a limit
|
||||
// nobody has reached on purpose is noise.
|
||||
if len(terms) > maxSearchTerms {
|
||||
terms = terms[:maxSearchTerms]
|
||||
}
|
||||
|
||||
var b strings.Builder
|
||||
for i, term := range terms {
|
||||
if i > 0 {
|
||||
@@ -66,10 +90,29 @@ func MatchExprOver(query string, columns ...string) string {
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// SearchTerms is the words a query asks about, capped the same way the MATCH
|
||||
// expression is.
|
||||
//
|
||||
// Exported because two searches need the same list: the FTS expression is built
|
||||
// from it, and so is the substring arm that finds a word living inside another
|
||||
// word. Two tokenizers would be two answers to "what did they type".
|
||||
func SearchTerms(query string) []string {
|
||||
terms := tokenize(query)
|
||||
if len(terms) > maxSearchTerms {
|
||||
terms = terms[:maxSearchTerms]
|
||||
}
|
||||
return terms
|
||||
}
|
||||
|
||||
// tokenize splits on everything that is not a letter or a digit. Splitting on
|
||||
// punctuation rather than whitespace alone means "q3/2026" searches for "q3" and
|
||||
// "2026" instead of for a term FTS5's own tokenizer would never have indexed.
|
||||
func tokenize(query string) []string {
|
||||
// Trimmed before it is split, so a megabyte of text is not walked rune by rune
|
||||
// only to have all but sixteen of the results thrown away.
|
||||
if len(query) > maxSearchInput {
|
||||
query = query[:maxSearchInput]
|
||||
}
|
||||
fields := strings.FieldsFunc(query, func(r rune) bool {
|
||||
return !unicode.IsLetter(r) && !unicode.IsDigit(r)
|
||||
})
|
||||
|
||||
@@ -124,3 +124,156 @@ func sealedStore(t *testing.T) (*DB, string) {
|
||||
}
|
||||
return db, user.ID
|
||||
}
|
||||
|
||||
// Three integrations, one person, and none of them may delete the others.
|
||||
//
|
||||
// `user_settings` had `user_id` as its whole primary key and an upsert that
|
||||
// rewrote `scope` on conflict, so every person had exactly one settings row and
|
||||
// the last write won the lot: saving an AI provider deleted the Gmail connection,
|
||||
// and connecting a calendar deleted the AI provider. The write reported success
|
||||
// each time, which is why it was reported as "the AI settings are not saved"
|
||||
// rather than as "connecting the calendar deleted them".
|
||||
//
|
||||
// Written against the store rather than the handlers, because the shape was the
|
||||
// bug: every caller was correct on its own.
|
||||
func TestOnePersonCanHaveSettingsForMoreThanOneThing(t *testing.T) {
|
||||
db, userID := sealedStore(t)
|
||||
ctx := context.Background()
|
||||
|
||||
want := map[string]map[string]any{
|
||||
"ai": {"provider": "anthropic", "model": "claude-sonnet-5", "api_key": "sk-ant-xxx"},
|
||||
"gmail": {"refresh_token": "1//0-gmail", "label": "Verdande"},
|
||||
"calendar": {"refresh_token": "1//0-calendar", "calendar_id": "primary"},
|
||||
}
|
||||
// In this order on purpose: the AI settings are written first and read last,
|
||||
// so a scope that steals the row is caught by the one written earliest.
|
||||
for _, scope := range []string{"ai", "gmail", "calendar"} {
|
||||
if err := db.SetUserSettings(ctx, userID, scope, want[scope]); err != nil {
|
||||
t.Fatalf("save %s: %v", scope, err)
|
||||
}
|
||||
}
|
||||
|
||||
for _, scope := range []string{"ai", "gmail", "calendar"} {
|
||||
got, err := db.UserSettings(ctx, userID, scope)
|
||||
if err != nil {
|
||||
t.Fatalf("read %s: %v", scope, err)
|
||||
}
|
||||
for key, value := range want[scope] {
|
||||
if got[key] != value {
|
||||
t.Errorf("%s.%s is %v, want %v", scope, key, got[key], value)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// And a second write to one scope updates it rather than adding another row —
|
||||
// the upsert still has to upsert.
|
||||
if err := db.SetUserSettings(ctx, userID, "ai", map[string]any{"provider": "openai"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var rows int
|
||||
if err := db.QueryRowContext(ctx,
|
||||
`SELECT count(*) FROM user_settings WHERE user_id = ?`, userID).Scan(&rows); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if rows != 3 {
|
||||
t.Errorf("the person has %d settings rows, want 3", rows)
|
||||
}
|
||||
back, err := db.UserSettings(ctx, userID, "ai")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if back["provider"] != "openai" {
|
||||
t.Errorf("the second write did not land: %v", back["provider"])
|
||||
}
|
||||
}
|
||||
|
||||
// The upgrade path, which is the one nobody runs: an existing database with a row
|
||||
// in the old shape, opened by a binary that carries 0025.
|
||||
//
|
||||
// Every other test here starts empty and runs all the migrations in order, which
|
||||
// exercises the migration's SQL but not its *situation*. The situation is a table
|
||||
// that already holds somebody's settings, and the question is whether they come
|
||||
// through it and whether a second scope can be written afterwards.
|
||||
func TestSettingsSurviveTheUpgradeToPerScopeRows(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "test.db")
|
||||
|
||||
db, err := Open(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx := context.Background()
|
||||
user := &User{Email: "[email protected]", Name: "Kristian"}
|
||||
if err := db.CreateUser(ctx, user, "Indbakke"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// Wind the database back to how v0.33.3 left it: user_id as the whole primary
|
||||
// key, and 0025 not yet applied.
|
||||
for _, stmt := range []string{
|
||||
`PRAGMA foreign_keys = OFF`,
|
||||
`CREATE TABLE us_old (
|
||||
user_id TEXT PRIMARY KEY REFERENCES users (id) ON DELETE CASCADE,
|
||||
scope TEXT NOT NULL DEFAULT 'general',
|
||||
values_json TEXT NOT NULL DEFAULT '{}',
|
||||
updated_at INTEGER NOT NULL
|
||||
)`,
|
||||
`INSERT INTO us_old (user_id, scope, values_json, updated_at)
|
||||
SELECT user_id, scope, values_json, updated_at FROM user_settings`,
|
||||
`DROP TABLE user_settings`,
|
||||
`ALTER TABLE us_old RENAME TO user_settings`,
|
||||
`DELETE FROM schema_migrations WHERE name = '0025_user_settings_per_scope.sql'`,
|
||||
`PRAGMA foreign_keys = ON`,
|
||||
} {
|
||||
if _, err := db.ExecContext(ctx, stmt); err != nil {
|
||||
t.Fatalf("rewind (%s): %v", stmt, err)
|
||||
}
|
||||
}
|
||||
|
||||
// A Gmail connection, written the way the old upsert wrote one.
|
||||
if _, err := db.ExecContext(ctx, `
|
||||
INSERT INTO user_settings (user_id, scope, values_json, updated_at)
|
||||
VALUES (?, 'gmail', '{"label":"Verdande"}', 0)`, user.ID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.Close(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// And now the new binary opens it.
|
||||
db, err = Open(path)
|
||||
if err != nil {
|
||||
t.Fatalf("the upgrade refused to open the database: %v", err)
|
||||
}
|
||||
defer db.Close()
|
||||
|
||||
gmail, err := db.UserSettings(ctx, user.ID, "gmail")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if gmail["label"] != "Verdande" {
|
||||
t.Errorf("the Gmail settings did not survive the migration: %v", gmail)
|
||||
}
|
||||
|
||||
// The thing that was reported: set up the AI provider, and have it still be
|
||||
// there afterwards.
|
||||
if err := db.SetUserSettings(ctx, user.ID, "ai", map[string]any{
|
||||
"provider": "anthropic", "model": "claude-sonnet-5",
|
||||
}); err != nil {
|
||||
t.Fatalf("saving the AI provider after the upgrade: %v", err)
|
||||
}
|
||||
ai, err := db.UserSettings(ctx, user.ID, "ai")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if ai["provider"] != "anthropic" {
|
||||
t.Errorf("the AI provider is %v after the upgrade, want anthropic", ai["provider"])
|
||||
}
|
||||
again, err := db.UserSettings(ctx, user.ID, "gmail")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if again["label"] != "Verdande" {
|
||||
t.Errorf("saving the AI provider took the Gmail settings: %v", again)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -166,15 +166,42 @@ func (db *DB) ListTasks(ctx context.Context, userID string, f TaskFilter) ([]Tas
|
||||
// labels are short, and their names are typed by the person searching for
|
||||
// them, so there is no spelling to be generous about. The diacritic
|
||||
// folding that matters for task text does not earn a second index here.
|
||||
//
|
||||
// The third arm is a word inside a word. An index is built of tokens, and
|
||||
// `SpeedAdmin` is one of them — so "speed admin" asks for two words that are
|
||||
// nowhere. A prefix query gets half of it (`speed*` matches, `admin*` cannot)
|
||||
// and the two are joined with AND, so the half that works is thrown away.
|
||||
//
|
||||
// Notes got the same arm, and for the same reason. Having it in one of the
|
||||
// two would be worse than having it in neither: "it finds it in my notes but
|
||||
// not in my tasks" is a bug report waiting, and the answer to it would be
|
||||
// this line.
|
||||
//
|
||||
// It scans, and it only ever runs when somebody typed something in the box —
|
||||
// Today and Kommende do not set `Search`. Every term has to appear, so the
|
||||
// arms are ANDed with each other while the whole thing is ORed with the two
|
||||
// above.
|
||||
inside := make([]string, 0, maxSearchTerms)
|
||||
insideArgs := make([]any, 0, maxSearchTerms)
|
||||
for _, term := range SearchTerms(f.Search) {
|
||||
inside = append(inside, "t.fold LIKE ?")
|
||||
insideArgs = append(insideArgs, "%"+FoldDanish(term)+"%")
|
||||
}
|
||||
text := ""
|
||||
if len(inside) > 0 {
|
||||
text = " OR (" + strings.Join(inside, " AND ") + ")"
|
||||
}
|
||||
|
||||
where = append(where, `(
|
||||
t.rowid IN (SELECT rowid FROM tasks_fts WHERE tasks_fts MATCH ?)
|
||||
OR t.id IN (
|
||||
SELECT tl.task_id FROM task_labels tl
|
||||
JOIN labels l ON l.id = tl.label_id
|
||||
WHERE lower(l.name) LIKE ?
|
||||
)
|
||||
)`+text+`
|
||||
)`)
|
||||
args = append(args, expr, "%"+strings.ToLower(strings.TrimSpace(f.Search))+"%")
|
||||
args = append(args, insideArgs...)
|
||||
}
|
||||
if f.FilterSQL != "" {
|
||||
where = append(where, "("+f.FilterSQL+")")
|
||||
|
||||
@@ -16,6 +16,11 @@ var (
|
||||
ErrNotFound = errors.New("store: not found")
|
||||
ErrEmailInUse = errors.New("store: that email address already has an account")
|
||||
ErrInviteInvalid = errors.New("store: invite is not valid")
|
||||
// ErrDuplicate er en unik nøgle, der blev brudt, oversat til noget en handler
|
||||
// kan spørge om. Drivereren udstiller ikke sin egen fejltype brugbart, og en
|
||||
// handler, der matcher på tekst, er en handler, der går i stykker den dag
|
||||
// driveren omformulerer sig.
|
||||
ErrDuplicate = errors.New("store: already exists")
|
||||
)
|
||||
|
||||
type User struct {
|
||||
@@ -36,8 +41,13 @@ type User struct {
|
||||
// is as true on the laptop as on the desktop. The sidebar's *width* is the
|
||||
// opposite case and is stored per browser.
|
||||
SidebarCollapsed []string
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
|
||||
// NavOrder is the order this person wants the fixed views in, by key. Empty
|
||||
// means the order the program ships with. Unknown keys are ignored and missing
|
||||
// ones are appended, so adding a view later is not a migration.
|
||||
NavOrder []string
|
||||
CreatedAt time.Time
|
||||
UpdatedAt time.Time
|
||||
}
|
||||
|
||||
// NormalizeEmail is the one place an address is canonicalised. Addresses arrive
|
||||
@@ -104,7 +114,7 @@ func (db *DB) UserByID(ctx context.Context, id string) (*User, error) {
|
||||
}
|
||||
|
||||
const userColumns = `id, email, name, password_hash, totp_secret, totp_enabled,
|
||||
avatar_color, timezone, locale, is_admin, sidebar_collapsed, created_at, updated_at`
|
||||
avatar_color, timezone, locale, is_admin, sidebar_collapsed, nav_order, created_at, updated_at`
|
||||
|
||||
func (db *DB) scanUser(ctx context.Context, query string, args ...any) (*User, error) {
|
||||
var u User
|
||||
@@ -112,11 +122,11 @@ func (db *DB) scanUser(ctx context.Context, query string, args ...any) (*User, e
|
||||
var totpEnabled, isAdmin int
|
||||
var created, updated int64
|
||||
|
||||
var collapsed string
|
||||
var collapsed, navOrder string
|
||||
|
||||
err := db.QueryRowContext(ctx, query, args...).Scan(
|
||||
&u.ID, &u.Email, &u.Name, &u.PasswordHash, &secret, &totpEnabled,
|
||||
&u.AvatarColor, &u.Timezone, &u.Locale, &isAdmin, &collapsed, &created, &updated)
|
||||
&u.AvatarColor, &u.Timezone, &u.Locale, &isAdmin, &collapsed, &navOrder, &created, &updated)
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return nil, ErrNotFound
|
||||
}
|
||||
@@ -124,18 +134,48 @@ func (db *DB) scanUser(ctx context.Context, query string, args ...any) (*User, e
|
||||
return nil, err
|
||||
}
|
||||
|
||||
u.TOTPSecret = secret.String
|
||||
// Opened on the way out. A row written before the key existed is returned as
|
||||
// itself — see secret.Unseal — so this does not need a migration; a secret
|
||||
// converts the next time it is written.
|
||||
if plain, err := db.unsealValue(secret.String); err == nil {
|
||||
u.TOTPSecret = plain
|
||||
} else {
|
||||
u.TOTPSecret = secret.String
|
||||
}
|
||||
u.TOTPEnabled = totpEnabled == 1
|
||||
u.IsAdmin = isAdmin == 1
|
||||
// A malformed value is read as "nothing folded" rather than as an error: the
|
||||
// worst case is a sidebar that opens fully, which is the state it shipped in.
|
||||
u.SidebarCollapsed = []string{}
|
||||
_ = json.Unmarshal([]byte(collapsed), &u.SidebarCollapsed)
|
||||
u.NavOrder = []string{}
|
||||
_ = json.Unmarshal([]byte(navOrder), &u.NavOrder)
|
||||
u.CreatedAt = time.Unix(created, 0).UTC()
|
||||
u.UpdatedAt = time.Unix(updated, 0).UTC()
|
||||
return &u, nil
|
||||
}
|
||||
|
||||
// SetNavOrder records the order of the fixed views.
|
||||
func (db *DB) SetNavOrder(ctx context.Context, userID string, order []string) error {
|
||||
if order == nil {
|
||||
order = []string{}
|
||||
}
|
||||
raw, err := json.Marshal(order)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
res, err := db.ExecContext(ctx,
|
||||
`UPDATE users SET nav_order = ?, updated_at = ? WHERE id = ?`,
|
||||
string(raw), time.Now().Unix(), userID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if n, _ := res.RowsAffected(); n == 0 {
|
||||
return ErrNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// SetSidebarCollapsed records which sidebar headings are folded.
|
||||
//
|
||||
// Its own method rather than a field on UpdateProfile: that one backs a form with
|
||||
@@ -405,9 +445,20 @@ func (db *DB) UpdateProfile(ctx context.Context, userID, name, timezone, locale
|
||||
}
|
||||
|
||||
func (db *DB) SetTOTPSecret(ctx context.Context, userID, secret string, enabled bool) error {
|
||||
// Sealed, like a mailbox password and for the same reason: the copies, not the
|
||||
// server. A TOTP secret in a downloadable backup lets whoever holds the copy
|
||||
// generate valid codes for every account on the instance — which turns the
|
||||
// second factor from a defence into a file somebody can take home.
|
||||
//
|
||||
// It was missed when internal/secret was written, along with three other
|
||||
// values; see SECURITY.md.
|
||||
var value any
|
||||
if secret != "" {
|
||||
value = secret
|
||||
sealed, err := db.sealValue(secret)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
value = sealed
|
||||
}
|
||||
_, err := db.ExecContext(ctx,
|
||||
`UPDATE users SET totp_secret = ?, totp_enabled = ?, updated_at = ? WHERE id = ?`,
|
||||
|
||||
@@ -6,12 +6,12 @@
|
||||
<title>verdande — self-hosted tasks, shared</title>
|
||||
<meta
|
||||
name="description"
|
||||
content="Self-hosted tasks and projects, shared. One Go binary with SQLite inside it. Your mail as an inbox, CalDAV, passkeys, and an MCP connector for Claude."
|
||||
content="Self-hosted tasks, notes and projects, shared. One Go binary with SQLite inside it. Your mail as an inbox, a CalDAV server, notes in Markdown, passkeys, and an MCP connector for Claude."
|
||||
/>
|
||||
<meta name="theme-color" content="#111312" />
|
||||
|
||||
<meta property="og:title" content="verdande" />
|
||||
<meta property="og:description" content="Self-hosted tasks and projects, shared. One binary, SQLite inside it." />
|
||||
<meta property="og:description" content="Self-hosted tasks, notes and projects, shared. One binary, SQLite inside it." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:image" content="screenshots/today.webp" />
|
||||
<meta property="og:image:alt" content="The Today view: projects and filters beside a list of what is due." />
|
||||
@@ -76,10 +76,10 @@
|
||||
}
|
||||
|
||||
.theme-toggle {
|
||||
position: fixed;
|
||||
top: 1rem;
|
||||
right: 1rem;
|
||||
z-index: 10;
|
||||
/* In the bar now rather than floating over the corner. Fixed, it sat
|
||||
on top of whatever happened to scroll under it — on a narrow window
|
||||
that was the first screenshot's own corner. */
|
||||
flex: none;
|
||||
width: 2.25rem;
|
||||
height: 2.25rem;
|
||||
display: grid;
|
||||
@@ -141,10 +141,86 @@
|
||||
padding: 0 1.5rem;
|
||||
}
|
||||
|
||||
/* --- the bar that stays -------------------------------------------
|
||||
|
||||
A page this long had its only way out at the very top: somebody who
|
||||
read as far as "What it will not do" had to scroll back past nine
|
||||
screenshots to reach the documentation. The bar is the way out from
|
||||
wherever you are.
|
||||
|
||||
`sticky` and not `fixed`: sticky keeps its space in the flow, so
|
||||
nothing below has to be pushed down by a number that then has to be
|
||||
kept in step with the bar's own height.
|
||||
|
||||
Full width, with the wrap *inside* it. Put the other way round, the
|
||||
background would end where the column ends and the page would scroll
|
||||
past it on both sides. */
|
||||
.topbar {
|
||||
position: sticky;
|
||||
top: 0;
|
||||
z-index: 20;
|
||||
background: var(--ground);
|
||||
border-bottom: 1px solid var(--line);
|
||||
}
|
||||
|
||||
.bar {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 1.25rem;
|
||||
padding: 0.625rem 1.5rem;
|
||||
}
|
||||
|
||||
.bar-brand {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.5rem;
|
||||
color: var(--ink);
|
||||
text-decoration: none;
|
||||
font-weight: 560;
|
||||
letter-spacing: -0.01em;
|
||||
}
|
||||
|
||||
.bar-brand .rune {
|
||||
color: var(--accent);
|
||||
font-size: 1.125rem;
|
||||
line-height: 1;
|
||||
}
|
||||
|
||||
.bar-links {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 1.25rem;
|
||||
margin-left: auto;
|
||||
font-size: 0.875rem;
|
||||
}
|
||||
|
||||
.bar-links a {
|
||||
color: var(--ink-muted);
|
||||
text-decoration: none;
|
||||
}
|
||||
|
||||
.bar-links a:hover {
|
||||
color: var(--ink);
|
||||
}
|
||||
|
||||
/* The three in-page links go first on a narrow screen: they are the ones
|
||||
a phone can reach by scrolling anyway. Documentation and Source cannot
|
||||
be reached any other way, so they stay. */
|
||||
@media (max-width: 640px) {
|
||||
.bar-links .inpage {
|
||||
display: none;
|
||||
}
|
||||
}
|
||||
|
||||
/* Or the bar covers the heading you just jumped to. */
|
||||
:target {
|
||||
scroll-margin-top: 4.5rem;
|
||||
}
|
||||
|
||||
/* --- hero --- */
|
||||
|
||||
header {
|
||||
padding: 6rem 0 4rem;
|
||||
.hero {
|
||||
padding: 5rem 0 3.5rem;
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
@@ -172,7 +248,7 @@
|
||||
}
|
||||
|
||||
.norn {
|
||||
margin: 2rem auto 0;
|
||||
margin: 0 auto 1.75rem;
|
||||
max-width: 34rem;
|
||||
font-size: 0.9375rem;
|
||||
color: var(--ink-faint);
|
||||
@@ -289,44 +365,104 @@
|
||||
|
||||
/* --- features --- */
|
||||
|
||||
/* A hairline between sections rather than under every heading: the page
|
||||
is one column, and the line says where one subject stops. */
|
||||
section {
|
||||
padding: 3rem 0;
|
||||
padding: 3.5rem 0;
|
||||
border-top: 1px solid var(--line);
|
||||
}
|
||||
|
||||
h2 {
|
||||
display: flex;
|
||||
align-items: baseline;
|
||||
gap: 0.75rem;
|
||||
font-size: 1.5rem;
|
||||
font-weight: 560;
|
||||
letter-spacing: -0.02em;
|
||||
margin: 0 0 2rem;
|
||||
}
|
||||
|
||||
/* A short accent rule in front of every h2. The sections had nothing to
|
||||
mark them but their own words, and at a glance the page read as one
|
||||
long run of paragraphs. */
|
||||
h2::before {
|
||||
content: '';
|
||||
flex: none;
|
||||
width: 1.125rem;
|
||||
height: 2px;
|
||||
border-radius: 1px;
|
||||
background: var(--accent);
|
||||
transform: translateY(-0.35em);
|
||||
}
|
||||
|
||||
/* Three groups rather than eighteen cards in a row. Grouped, the list is
|
||||
read as three answers — what it does, what it talks to, what it
|
||||
promises — and any of the three can be skipped. */
|
||||
.group-label {
|
||||
margin: 2.5rem 0 1rem;
|
||||
font-size: 0.75rem;
|
||||
font-weight: 560;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.08em;
|
||||
color: var(--ink-faint);
|
||||
}
|
||||
|
||||
.group-label:first-of-type {
|
||||
margin-top: 0;
|
||||
}
|
||||
|
||||
/* Cards with their own edges, not cells in one table.
|
||||
The table shared a hairline grid, which meant the last row had to be
|
||||
full or it was a hole in the drawing — and it was a hole, for as long
|
||||
as there were ten of them. A card that stands alone on its row is a
|
||||
card; an empty cell is a mistake. */
|
||||
.grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fit, minmax(260px, 1fr));
|
||||
gap: 1px;
|
||||
background: var(--line);
|
||||
border: 1px solid var(--line);
|
||||
border-radius: var(--radius-lg);
|
||||
overflow: hidden;
|
||||
grid-template-columns: repeat(auto-fit, minmax(15rem, 1fr));
|
||||
gap: 0.875rem;
|
||||
}
|
||||
|
||||
.cell {
|
||||
background: var(--ground);
|
||||
padding: 1.5rem;
|
||||
background: var(--surface);
|
||||
border: 1px solid var(--line);
|
||||
border-radius: var(--radius-lg);
|
||||
padding: 1.125rem 1.25rem 1.25rem;
|
||||
transition: border-color 150ms ease;
|
||||
}
|
||||
|
||||
.cell:hover {
|
||||
border-color: var(--line-strong);
|
||||
}
|
||||
|
||||
.cell h3 {
|
||||
margin: 0 0 0.5rem;
|
||||
font-size: 1rem;
|
||||
margin: 0 0 0.4375rem;
|
||||
font-size: 0.9375rem;
|
||||
font-weight: 560;
|
||||
}
|
||||
|
||||
.cell p {
|
||||
margin: 0;
|
||||
font-size: 0.9375rem;
|
||||
font-size: 0.875rem;
|
||||
color: var(--ink-muted);
|
||||
}
|
||||
|
||||
.cell code {
|
||||
font-family: var(--font-mono);
|
||||
font-size: 0.8125em;
|
||||
color: var(--ink);
|
||||
}
|
||||
|
||||
/* A card that has something to admit rather than something to offer.
|
||||
It is not a footnote and it is not fine print: it is the same size as
|
||||
everything else, and it says the same thing the README says. */
|
||||
.cell .caveat {
|
||||
margin: 0.5rem 0 0;
|
||||
padding-top: 0.5rem;
|
||||
border-top: 1px solid var(--line);
|
||||
font-size: 0.8125rem;
|
||||
color: var(--ink-faint);
|
||||
}
|
||||
|
||||
/* --- install --- */
|
||||
|
||||
pre {
|
||||
@@ -374,6 +510,168 @@
|
||||
color: var(--ink-muted);
|
||||
}
|
||||
|
||||
/* Four facts under the buttons, in the place where a landing page usually
|
||||
puts logos it has not earned. They are the four questions somebody
|
||||
self-hosting asks before anything else. */
|
||||
ul.facts {
|
||||
list-style: none;
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
justify-content: center;
|
||||
gap: 0.5rem 1.25rem;
|
||||
margin: 2rem 0 0;
|
||||
padding: 0;
|
||||
font-size: 0.8125rem;
|
||||
color: var(--ink-faint);
|
||||
}
|
||||
|
||||
ul.facts li {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 1.25rem;
|
||||
}
|
||||
|
||||
/* The separator belongs between two facts, not after the last one — so it
|
||||
is drawn by the item that follows it. */
|
||||
ul.facts li + li::before {
|
||||
content: '';
|
||||
width: 3px;
|
||||
height: 3px;
|
||||
border-radius: 50%;
|
||||
background: var(--line-strong);
|
||||
}
|
||||
|
||||
/* On a phone the row wraps, and a separator drawn by the item that follows
|
||||
it lands at the *start* of the new line — a dot with nothing before it.
|
||||
No separators there; the wrap is the separation. */
|
||||
@media (max-width: 640px) {
|
||||
ul.facts {
|
||||
gap: 0.375rem 1.125rem;
|
||||
}
|
||||
|
||||
ul.facts li + li::before {
|
||||
display: none;
|
||||
}
|
||||
}
|
||||
|
||||
/* Two ways to run it, side by side, because they are alternatives rather
|
||||
than steps. Stacked, the second read as what you do after the first. */
|
||||
.panels {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fit, minmax(19rem, 1fr));
|
||||
gap: 0.875rem;
|
||||
}
|
||||
|
||||
.panel {
|
||||
background: var(--surface);
|
||||
border: 1px solid var(--line);
|
||||
border-radius: var(--radius-lg);
|
||||
padding: 1.25rem;
|
||||
}
|
||||
|
||||
.panel h3 {
|
||||
margin: 0 0 0.5rem;
|
||||
font-size: 0.9375rem;
|
||||
font-weight: 560;
|
||||
}
|
||||
|
||||
/* Sunk rather than raised: inside a card, a block in --surface would be
|
||||
the same colour as the card it sits in. */
|
||||
.panel pre {
|
||||
background: var(--ground);
|
||||
margin: 1rem 0 0;
|
||||
}
|
||||
|
||||
.panel p {
|
||||
margin: 0;
|
||||
font-size: 0.875rem;
|
||||
color: var(--ink-muted);
|
||||
}
|
||||
|
||||
/* Versionsnummeret i sin egen søjle, så øjet kan løbe ned ad dem uden at
|
||||
læse sætningerne. `ch` frem for pixels: søjlen er lige så bred som de
|
||||
tal, der står i den, og den følger med, hvis skriften skifter. */
|
||||
ul.lately {
|
||||
list-style: none;
|
||||
margin: 1.5rem 0 0;
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
ul.lately li {
|
||||
display: grid;
|
||||
grid-template-columns: 5ch 1fr;
|
||||
gap: 0 1rem;
|
||||
padding: 0.875rem 0;
|
||||
border-top: 1px solid var(--line);
|
||||
font-size: 0.9375rem;
|
||||
color: var(--ink-muted);
|
||||
}
|
||||
|
||||
ul.lately .ver {
|
||||
font-family: var(--font-mono);
|
||||
font-size: 0.8125rem;
|
||||
color: var(--ink-faint);
|
||||
/* Tallet står på linje med den første tekstlinje og ikke over den. */
|
||||
line-height: 1.7;
|
||||
}
|
||||
|
||||
@media (max-width: 520px) {
|
||||
ul.lately li {
|
||||
grid-template-columns: 1fr;
|
||||
gap: 0.25rem;
|
||||
}
|
||||
}
|
||||
|
||||
/* Risikoen står i kort, ikke i småt.
|
||||
Et forbehold sat med mindre skrift end resten er et forbehold, der er
|
||||
skrevet for at kunne peges på bagefter — og det er den ene ting, denne
|
||||
side ikke skal gøre. Samme størrelse som alt andet. */
|
||||
.risk {
|
||||
border: 1px solid var(--line-strong);
|
||||
border-radius: var(--radius-lg);
|
||||
padding: 1.5rem;
|
||||
background: var(--surface);
|
||||
}
|
||||
|
||||
.risk p {
|
||||
margin: 0 0 1rem;
|
||||
font-size: 0.9375rem;
|
||||
color: var(--ink-muted);
|
||||
}
|
||||
|
||||
.risk p:last-child {
|
||||
margin-bottom: 0;
|
||||
}
|
||||
|
||||
.risk strong {
|
||||
color: var(--ink);
|
||||
}
|
||||
|
||||
/* The non-goals were one paragraph of seven sentences, which is the shape
|
||||
nobody reads. Each is its own line now, and they can be counted. */
|
||||
ul.nogo {
|
||||
list-style: none;
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fit, minmax(16rem, 1fr));
|
||||
gap: 0.5rem 1.75rem;
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
font-size: 0.9375rem;
|
||||
color: var(--ink-muted);
|
||||
}
|
||||
|
||||
ul.nogo li {
|
||||
position: relative;
|
||||
padding-left: 1.25rem;
|
||||
}
|
||||
|
||||
ul.nogo li::before {
|
||||
content: '—';
|
||||
position: absolute;
|
||||
left: 0;
|
||||
color: var(--ink-faint);
|
||||
}
|
||||
|
||||
.demo-note {
|
||||
margin: 1rem 0 0;
|
||||
font-family: var(--font);
|
||||
@@ -524,27 +822,38 @@
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<button class="theme-toggle" type="button" aria-label="Switch between light and dark" title="Light or dark">
|
||||
<span aria-hidden="true">◐</span>
|
||||
</button>
|
||||
<div class="wrap">
|
||||
<header>
|
||||
<div class="topbar">
|
||||
<div class="wrap bar">
|
||||
<a class="bar-brand" href="#top">
|
||||
<span class="rune" aria-hidden="true">ᚹ</span>
|
||||
verdande
|
||||
</a>
|
||||
<nav class="bar-links" aria-label="Sections">
|
||||
<a class="inpage" href="#screens">Screens</a>
|
||||
<a class="inpage" href="#features">Features</a>
|
||||
<a class="inpage" href="#running">Running it</a>
|
||||
<a href="https://kristianwind.github.io/verdande/">Documentation</a>
|
||||
<a href="https://github.com/kristianwind/verdande">Source</a>
|
||||
<button class="theme-toggle" type="button" aria-label="Switch between light and dark" title="Light or dark">
|
||||
<span aria-hidden="true">◐</span>
|
||||
</button>
|
||||
</nav>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="wrap" id="top">
|
||||
<header class="hero">
|
||||
<span class="mark" aria-hidden="true">ᚹ</span>
|
||||
<h1>verdande</h1>
|
||||
<p class="lede">
|
||||
Self-hosted tasks and projects, shared with the people you share the work with.
|
||||
One binary, SQLite inside it.
|
||||
Self-hosted tasks, notes and projects, shared with the people you share the work
|
||||
with. One binary, SQLite inside it.
|
||||
</p>
|
||||
<p class="rune-note">
|
||||
Built to be deployed as a <strong>Rune</strong> in
|
||||
<a href="https://github.com/kristianwind/yggdrasil">Yggdrasil Panel</a> — upload
|
||||
the manifest, fill in two fields, press start. It also runs anywhere Docker does.
|
||||
</p>
|
||||
<p class="norn">
|
||||
Verdande is one of the three Norns — the one who spins <em>that which is
|
||||
becoming</em>. Not what has been, and not what is fated to be: the thing you are
|
||||
in the middle of. Which is what a to-do list is.
|
||||
</p>
|
||||
|
||||
<div class="actions">
|
||||
<a class="button primary" href="https://kristianwind.github.io/verdande/self-hosting/">
|
||||
@@ -553,6 +862,14 @@
|
||||
<a class="button" href="https://github.com/kristianwind/verdande">Source</a>
|
||||
<a class="button" href="https://kristianwind.github.io/verdande/">Documentation</a>
|
||||
</div>
|
||||
|
||||
<ul class="facts">
|
||||
<li>One Go binary</li>
|
||||
<li>SQLite inside it</li>
|
||||
<li>amd64 and arm64</li>
|
||||
<li>No account anywhere but yours</li>
|
||||
<li>MIT</li>
|
||||
</ul>
|
||||
</header>
|
||||
|
||||
<!-- No screenshot: the quick-add parser is the thing worth showing, and it
|
||||
@@ -584,7 +901,7 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<section>
|
||||
<section id="screens">
|
||||
<h2>What it looks like</h2>
|
||||
<figure class="shot only-dark">
|
||||
<img
|
||||
@@ -621,13 +938,13 @@
|
||||
</figure>
|
||||
<figure class="shot">
|
||||
<img
|
||||
src="screenshots/today-light.webp"
|
||||
alt="The same Today view in the light theme."
|
||||
src="screenshots/notes.webp"
|
||||
alt="Notes: a list of notes on the left, one open on the right with headings, bullet lists, a quotation and a shell command in a code block."
|
||||
width="1400"
|
||||
height="900"
|
||||
loading="lazy"
|
||||
/>
|
||||
<figcaption>The light theme.</figcaption>
|
||||
<figcaption>Notes. Rich text on top, Markdown underneath.</figcaption>
|
||||
</figure>
|
||||
<figure class="shot">
|
||||
<img
|
||||
@@ -642,7 +959,7 @@
|
||||
<figure class="shot">
|
||||
<img
|
||||
src="screenshots/detail.webp"
|
||||
alt="A task open in the detail drawer, with its description, due date, labels, sub-tasks and comments."
|
||||
alt="A task open in the detail drawer, with its description, priority, due date, labels, sub-tasks and an attached file."
|
||||
width="1400"
|
||||
height="900"
|
||||
loading="lazy"
|
||||
@@ -652,12 +969,12 @@
|
||||
<figure class="shot">
|
||||
<img
|
||||
src="screenshots/themes.webp"
|
||||
alt="The settings page showing the five themes as swatches."
|
||||
alt="The settings page showing five themes as swatches and, below them, four looks written in the typeface each one selects."
|
||||
width="1400"
|
||||
height="900"
|
||||
loading="lazy"
|
||||
/>
|
||||
<figcaption>Five themes, all of them measured for contrast.</figcaption>
|
||||
<figcaption>Five themes and four looks: how bright, and how it reads.</figcaption>
|
||||
</figure>
|
||||
<figure class="shot">
|
||||
<img
|
||||
@@ -669,11 +986,43 @@
|
||||
/>
|
||||
<figcaption>What has been done on the server, across every project.</figcaption>
|
||||
</figure>
|
||||
<figure class="shot">
|
||||
<img
|
||||
src="screenshots/delegated.webp"
|
||||
alt="Waiting on others: tasks that have been handed to somebody else, grouped under the person who has them."
|
||||
width="1400"
|
||||
height="900"
|
||||
loading="lazy"
|
||||
/>
|
||||
<figcaption>Waiting on others — grouped by who has it.</figcaption>
|
||||
</figure>
|
||||
<figure class="shot">
|
||||
<img
|
||||
src="screenshots/notes-source.webp"
|
||||
alt="A note with the source view open: the same note as Markdown, with headings, a list, a quotation, a wiki link and a fenced code block."
|
||||
width="1400"
|
||||
height="900"
|
||||
loading="lazy"
|
||||
/>
|
||||
<figcaption>The same note as Markdown. What is on disk is what you see.</figcaption>
|
||||
</figure>
|
||||
<figure class="shot">
|
||||
<img
|
||||
src="screenshots/settings.webp"
|
||||
alt="The integrations settings page: connecting Gmail with an OAuth client, and below it the calendar feed address."
|
||||
width="1400"
|
||||
height="900"
|
||||
loading="lazy"
|
||||
/>
|
||||
<figcaption>Integrations: your mailbox, your calendar feed, your address.</figcaption>
|
||||
</figure>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<section id="features">
|
||||
<h2>What it does</h2>
|
||||
|
||||
<p class="group-label">Doing the work</p>
|
||||
<div class="grid">
|
||||
<div class="cell">
|
||||
<h3>Reads what you type</h3>
|
||||
@@ -682,6 +1031,50 @@
|
||||
one line — in Danish and English, mixed freely.
|
||||
</p>
|
||||
</div>
|
||||
<div class="cell">
|
||||
<h3>Nothing to remember</h3>
|
||||
<p>
|
||||
Start typing anywhere and you are writing a task, with the letter you
|
||||
pressed already in the field. <code>⌘K</code> goes anywhere else.
|
||||
</p>
|
||||
</div>
|
||||
<div class="cell">
|
||||
<h3>Search that reads Danish</h3>
|
||||
<p>
|
||||
Type <code>gron</code> and find <code>grøn</code>. Unicode does not
|
||||
consider <code>ø</code> an accented <code>o</code>, so a search that
|
||||
leans on that alone is broken for the people this was built for.
|
||||
</p>
|
||||
</div>
|
||||
<div class="cell">
|
||||
<h3>Repeats, and says so in words</h3>
|
||||
<p>
|
||||
Recurring tasks are RRULE underneath and "every other Tuesday" on the
|
||||
screen. Reminders are absolute or relative, and go out to the devices
|
||||
you have allowed them on.
|
||||
</p>
|
||||
</div>
|
||||
<div class="cell">
|
||||
<h3>Notes, beside the work</h3>
|
||||
<p>
|
||||
Rich text to write in, Markdown on disk — and a source view to see it.
|
||||
Images are pasted or dragged in. <code>#project</code> and
|
||||
<code>[[another note]]</code> are links, and a task shows what has been
|
||||
written about it. Archive what is done with it; the bin is for mistakes.
|
||||
</p>
|
||||
</div>
|
||||
<div class="cell">
|
||||
<h3>Waiting on others</h3>
|
||||
<p>
|
||||
What you have handed to somebody else, on its own page, grouped under
|
||||
the person who has it. The thing a shared list stops being able to tell
|
||||
you the moment it is shared.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<p class="group-label">Connected to what you already use</p>
|
||||
<div class="grid">
|
||||
<div class="cell">
|
||||
<h3>Shared, properly</h3>
|
||||
<p>
|
||||
@@ -696,6 +1089,27 @@
|
||||
here. Not a read-only feed pretending.
|
||||
</p>
|
||||
</div>
|
||||
<div class="cell">
|
||||
<h3>A calendar of its own</h3>
|
||||
<p>
|
||||
A month or a week of what is due, and a secret ICS address to subscribe
|
||||
to from Apple Calendar, Google or Thunderbird. A Google calendar can be
|
||||
laid over the top, read-only.
|
||||
</p>
|
||||
<p class="caveat">
|
||||
The Google half is written to Google's documentation and has not yet
|
||||
been run against Google. The feed and CalDAV have.
|
||||
</p>
|
||||
</div>
|
||||
<div class="cell">
|
||||
<h3>Your mail, as an inbox</h3>
|
||||
<p>
|
||||
Star a mail in Gmail, or flag one over IMAP in iCloud or Fastmail, and
|
||||
it becomes a task. Forward one to your own address instead and the
|
||||
subject is parsed like anything else you type. One way: unstarring it
|
||||
later does nothing to the task you made.
|
||||
</p>
|
||||
</div>
|
||||
<div class="cell">
|
||||
<h3>Claude, connected</h3>
|
||||
<p>
|
||||
@@ -704,35 +1118,39 @@
|
||||
</p>
|
||||
</div>
|
||||
<div class="cell">
|
||||
<h3>Your mail, as an inbox</h3>
|
||||
<h3>AI only if you ask for it</h3>
|
||||
<p>
|
||||
Star a mail in Gmail, or flag one in iCloud or Fastmail, and it becomes
|
||||
a task. Forward one to your own address instead and the subject is
|
||||
parsed like anything else you type. One way: unstarring it later does
|
||||
nothing to the task you made.
|
||||
Off by default, your own key, and your choice of Anthropic, OpenAI,
|
||||
Google or something local. Nothing leaves the server until you have
|
||||
filled that in, and everything works without it.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<p class="group-label">Yours, and checkable</p>
|
||||
<div class="grid">
|
||||
<div class="cell">
|
||||
<h3>Nothing to remember</h3>
|
||||
<h3>More than one way in</h3>
|
||||
<p>
|
||||
Start typing anywhere and you are writing a task, with the letter you
|
||||
pressed already in the field. Sign in with a passkey and there is no
|
||||
password to forget either.
|
||||
</p>
|
||||
</div>
|
||||
<div class="cell">
|
||||
<h3>Yours to leave</h3>
|
||||
<p>
|
||||
Import from Todoist, export back to it, or take everything as JSON. The
|
||||
round trip is tested, not promised.
|
||||
Passkeys, or a password hashed with Argon2id and a second factor with
|
||||
recovery codes. Every device you are signed in on is listed, and any of
|
||||
them can be signed out from here.
|
||||
</p>
|
||||
</div>
|
||||
<div class="cell">
|
||||
<h3>Keys stay behind</h3>
|
||||
<p>
|
||||
Mail tokens and mailbox passwords are encrypted in the database, and the
|
||||
key lives in a file beside it — not inside it. So a backup you download,
|
||||
or one that ends up somewhere it should not, does not open your mail.
|
||||
Mail tokens, mailbox passwords, second-factor secrets and AI keys are
|
||||
encrypted in the database, and the key lives in a file beside it — not
|
||||
inside it. So a backup you download, or one that ends up somewhere it
|
||||
should not, does not open your mail.
|
||||
</p>
|
||||
</div>
|
||||
<div class="cell">
|
||||
<h3>Nobody signs themselves up</h3>
|
||||
<p>
|
||||
The first account is the administrator, and after that people arrive by
|
||||
invitation. An address on the internet is not an open door.
|
||||
</p>
|
||||
</div>
|
||||
<div class="cell">
|
||||
@@ -743,50 +1161,157 @@
|
||||
instance's record, not the person's.
|
||||
</p>
|
||||
</div>
|
||||
<div class="cell">
|
||||
<h3>Yours to leave</h3>
|
||||
<p>
|
||||
Import from Todoist, export back to it, take everything as JSON, or
|
||||
hand the notes a zip of Markdown and get the same zip back out. The
|
||||
round trip is tested, not promised.
|
||||
</p>
|
||||
</div>
|
||||
<div class="cell">
|
||||
<h3>It counts itself, out loud</h3>
|
||||
<p>
|
||||
Once a day an instance says it exists — a random id, the version, and
|
||||
nothing else. No tasks, no names, no addresses. It is on to begin with,
|
||||
the settings page prints the exact payload, and one switch stops it.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<section id="recently">
|
||||
<h2>Lately</h2>
|
||||
<p class="muted">
|
||||
A landing page can say a project is alive; a list of what landed says it
|
||||
better. This is not a changelog — every release has notes, and they are on
|
||||
<a href="https://github.com/kristianwind/verdande/releases">the releases page</a>.
|
||||
</p>
|
||||
<ul class="lately">
|
||||
<li>
|
||||
<span class="ver">0.34</span>
|
||||
Sections can be dragged into the order the work actually happens in. The
|
||||
sidebar keeps its head still while the projects scroll.
|
||||
</li>
|
||||
<li>
|
||||
<span class="ver">0.33</span>
|
||||
Notes grew an archive, selection of many at once, pictures you can paste
|
||||
or drag in, a source view, and a copy button on code blocks. Search
|
||||
stopped answering by date and started answering by relevance.
|
||||
</li>
|
||||
<li>
|
||||
<span class="ver">0.32</span>
|
||||
A calendar of its own as a menu item, with a read-only Google calendar
|
||||
that can be laid over it.
|
||||
</li>
|
||||
<li>
|
||||
<span class="ver">0.31</span>
|
||||
Mailboxes over IMAP, so flagging a mail in iCloud or Fastmail makes a
|
||||
task — not only Gmail.
|
||||
</li>
|
||||
<li>
|
||||
<span class="ver">0.30</span>
|
||||
An installation count that says exactly what it sends, and can be turned
|
||||
off in one click.
|
||||
</li>
|
||||
</ul>
|
||||
</section>
|
||||
|
||||
<section id="running">
|
||||
<h2>Running it</h2>
|
||||
|
||||
<h3 class="sub">As a Rune in Yggdrasil Panel</h3>
|
||||
<p class="muted">
|
||||
This is what it was built for. <strong>Runes → Carve a rune</strong>, upload
|
||||
<code>rune/verdande.yaml</code>, create a server from it, set the public address
|
||||
and your mail server, and start it. No port forwarding — give it a subdomain and
|
||||
the panel routes it through your Cloudflare Tunnel.
|
||||
</p>
|
||||
|
||||
<h3 class="sub">Or with Docker, anywhere</h3>
|
||||
<pre><code>docker run -d --name verdande \
|
||||
<div class="panels">
|
||||
<div class="panel">
|
||||
<h3>As a Rune in Yggdrasil Panel</h3>
|
||||
<p>
|
||||
This is what it was built for. <strong>Runes → Carve a rune</strong>,
|
||||
upload <code>rune/verdande.yaml</code>, create a server from it, set the
|
||||
public address and your mail server, and start it. No port forwarding —
|
||||
give it a subdomain and the panel routes it through your Cloudflare
|
||||
Tunnel.
|
||||
</p>
|
||||
</div>
|
||||
<div class="panel">
|
||||
<h3>Or with Docker, anywhere</h3>
|
||||
<p>Open the address afterwards and create the first account. That one is the administrator.</p>
|
||||
<pre><code>docker run -d --name verdande \
|
||||
-p 8080:8080 \
|
||||
-v verdande-data:/data \
|
||||
-e VERDANDE_BASE_URL=https://todo.example.dk \
|
||||
ghcr.io/kristianwind/verdande:latest</code></pre>
|
||||
<p class="muted" style="margin-top: 1rem">
|
||||
Open the address and create the first account. That one is the administrator.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<ul class="plain">
|
||||
<li>No database to provision — SQLite is inside the binary.</li>
|
||||
<li>Distroless image: no shell, no package manager, one static file.</li>
|
||||
<li>amd64 and arm64, so a Raspberry Pi is fine.</li>
|
||||
<li>Nightly backups, fourteen kept, written with <code>VACUUM INTO</code>.</li>
|
||||
<li>Twenty-four migrations, run on start. Upgrading is pulling the new image.</li>
|
||||
</ul>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<section id="limits">
|
||||
<h2>What it will not do</h2>
|
||||
<p class="muted">
|
||||
No karma or points. No team workspaces. No native apps. No mail integration
|
||||
beyond Gmail and the forwarding address. No languages beyond Danish and English.
|
||||
And no sync protocol of its own — CalDAV and the API <em>are</em> the sync.
|
||||
</p>
|
||||
<ul class="nogo">
|
||||
<li>No karma, streaks or points.</li>
|
||||
<li>No team workspaces — projects and the people in them are the whole model.</li>
|
||||
<li>No native apps. It is a PWA, and it installs on a phone.</li>
|
||||
<li>No Outlook or Exchange: Microsoft has closed IMAP to passwords and wants its own flow.</li>
|
||||
<li>No languages beyond Danish and English.</li>
|
||||
<li>No sync protocol of its own — CalDAV and the API <em>are</em> the sync.</li>
|
||||
<li>No hosted version. There is nowhere to sign up, and that is the point.</li>
|
||||
</ul>
|
||||
</section>
|
||||
|
||||
<section id="risk">
|
||||
<h2>At your own risk</h2>
|
||||
<div class="risk">
|
||||
<p>
|
||||
verdande is <strong>MIT licensed and provided as is, without warranty of
|
||||
any kind</strong>. That sentence is in every MIT licence and it is usually
|
||||
skipped, so here it is in the open: if you run this, you run it at your
|
||||
own risk, and nobody is on the hook if it loses your work.
|
||||
</p>
|
||||
<p>
|
||||
It is one person's project, at version 0.x, and 0.x means the shape can
|
||||
still change. <strong>You host it, so the data and the backups are
|
||||
yours.</strong> It writes a backup every night and keeps fourteen — and a
|
||||
backup nobody has ever restored is not yet a backup. Restore one before
|
||||
you need to.
|
||||
</p>
|
||||
<p>
|
||||
Security is worked on rather than claimed. There have been two full
|
||||
reviews of the code, every finding is written down and closed, and what is
|
||||
protected and what is not is spelled out in
|
||||
<a href="https://github.com/kristianwind/verdande/blob/main/SECURITY.md">SECURITY.md</a>
|
||||
— including the parts that are a deliberate trade-off. <strong>Nobody
|
||||
outside the project has audited it.</strong> The last review found two ways
|
||||
to reach something you should not and closed both; a third review will find
|
||||
more, which is the point of having one.
|
||||
</p>
|
||||
<p>
|
||||
It is built to sit behind something — a tunnel, a reverse proxy, a VPN —
|
||||
put there by somebody who knows what they exposed. Found a hole? Mail it
|
||||
rather than opening an issue; the address is in SECURITY.md and you get an
|
||||
answer within a week.
|
||||
</p>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<footer>
|
||||
<!-- The Norn belongs at the end. It was the third paragraph of the hero,
|
||||
where it stood between somebody arriving and the buttons they came
|
||||
for — and it is a closing thought, not an opening one. -->
|
||||
<p class="norn">
|
||||
Verdande is one of the three Norns — the one who spins <em>that which is
|
||||
becoming</em>. Not what has been, and not what is fated to be: the thing you are
|
||||
in the middle of. Which is what a to-do list is.
|
||||
</p>
|
||||
<p>
|
||||
MIT licensed ·
|
||||
<a href="https://github.com/kristianwind/verdande">github.com/kristianwind/verdande</a>
|
||||
· <a href="https://kristianwind.github.io/verdande/">Documentation</a>
|
||||
</p>
|
||||
<p>
|
||||
Built to run as a Rune in
|
||||
|
||||
|
Before Width: | Height: | Size: 32 KiB After Width: | Height: | Size: 40 KiB |
|
Before Width: | Height: | Size: 34 KiB After Width: | Height: | Size: 44 KiB |
|
Before Width: | Height: | Size: 16 KiB After Width: | Height: | Size: 22 KiB |
|
Before Width: | Height: | Size: 37 KiB After Width: | Height: | Size: 47 KiB |
|
Before Width: | Height: | Size: 45 KiB After Width: | Height: | Size: 40 KiB |
|
After Width: | Height: | Size: 56 KiB |
|
After Width: | Height: | Size: 52 KiB |
|
Before Width: | Height: | Size: 31 KiB After Width: | Height: | Size: 35 KiB |