142 lines
5.6 KiB
Markdown
142 lines
5.6 KiB
Markdown
# go-fa-api
|
|
|
|
Go 1.25+ SDK for [FurAffinity](https://www.furaffinity.net). FA has no JSON
|
|
API, so this library scrapes the beta theme via [Colly](https://github.com/gocolly/colly)
|
|
internally and exposes a strongly typed surface.
|
|
|
|
## Features
|
|
|
|
| Area | What you get |
|
|
|------|--------------|
|
|
| **Read** | `GetSubmission` · `GetUser` · `GetJournal` · `Download` |
|
|
| **Iterators** (`iter.Seq2`) | `Gallery` · `Scraps` · `Favorites` · `UserJournals` · `SubmissionComments` · `JournalComments` |
|
|
| **Discovery** | `Browse(BrowseOptions)` · `Search(query, SearchOptions)` full filter struct (rating/type/order/range/etc.) |
|
|
| **Inbox** | `SubmissionInbox` (watched-users feed) · `Notes` · `GetNote` · `Notifications` (journals/watches/comments/favs/shouts) |
|
|
| **Write** | `Fav`/`Unfav` · `Watch`/`Unwatch` · `PostSubmissionComment` · `PostJournalComment` · `SendNote` |
|
|
| **Auth** | `WithCookies(a, b)` · `WithCloudflare(cf_clearance)` · `WithUserAgent` · `WithSFW(SFWOn/Off/Auto)` |
|
|
| **Networking** | Built-in 1 req/s rate limiter · Cloudflare challenge detection · 429 `Retry-After` · 5xx exponential backoff · `context.Context` end-to-end |
|
|
| **Experimental** | `WithExperimentalJSONListings(true)` listing pages merge from FA's embedded `js-submissionData` JSON first, HTML fallback if it's absent. More resilient to markup drift; off by default. |
|
|
|
|
Pre-1.0: expect breaking changes. Beta theme only; classic theme is not parsed.
|
|
|
|
## Install
|
|
|
|
```bash
|
|
go get git.anthrove.art/public/go-fa-api
|
|
```
|
|
|
|
## Quickstart
|
|
|
|
```go
|
|
package main
|
|
|
|
import (
|
|
"context"
|
|
"fmt"
|
|
"log"
|
|
"os"
|
|
|
|
fa "git.anthrove.art/public/go-fa-api"
|
|
)
|
|
|
|
func main() {
|
|
client := fa.New(
|
|
fa.WithCookies(fa.Cookies{A: os.Getenv("FA_A"), B: os.Getenv("FA_B")}),
|
|
fa.WithCloudflare(fa.CFCookies{Clearance: os.Getenv("CF_CLEARANCE")}),
|
|
fa.WithUserAgent(os.Getenv("FA_UA")),
|
|
)
|
|
|
|
sub, err := client.GetSubmission(context.Background(), 12345678)
|
|
if err != nil {
|
|
log.Fatal(err)
|
|
}
|
|
fmt.Printf("%s by %s\n", sub.Title, sub.Author.DisplayName)
|
|
|
|
for sub, err := range client.Gallery(context.Background(), "someuser", fa.ListOptions{MaxPages: 3}) {
|
|
if err != nil {
|
|
log.Fatal(err)
|
|
}
|
|
fmt.Println(sub.ID, sub.Title)
|
|
}
|
|
}
|
|
```
|
|
|
|
Runnable examples in `examples/`: `basic`, `gallery_dump`, `download`, `browse`, `search`, `inbox`, `notes`, `notifications`.
|
|
|
|
## Auth
|
|
|
|
FA's auth is cookie-based; Cloudflare gates submissions behind a clearance
|
|
cookie bound to your browser's User-Agent. The SDK doesn't log in for you copy the values once:
|
|
|
|
1. Log in at https://www.furaffinity.net in your browser.
|
|
2. DevTools → **Application** → **Cookies** → copy `a`, `b`, and `cf_clearance`.
|
|
3. DevTools → **Network** → any FA request → copy the **User-Agent** header
|
|
*exactly*. CF binds `cf_clearance` to that UA; mismatch = challenge.
|
|
4. Pass them in via `fa.WithCookies` / `fa.WithCloudflare` / `fa.WithUserAgent`.
|
|
|
|
If FA challenges you later, your `cf_clearance` expired refresh it and the
|
|
SDK will return `fa.ErrCloudflareChallenge` so you can detect it.
|
|
|
|
> Account preference must be set to the **beta** theme (Site Preferences → Style → Beta).
|
|
|
|
## Rate limiting
|
|
|
|
Every request (page fetches *and* downloads) passes through one token bucket.
|
|
Default is **1 req/s**, lowest safe for FA. Lives inside the
|
|
`http.RoundTripper` so callers can't bypass it.
|
|
|
|
```go
|
|
fa.New(fa.WithRateLimit(2*time.Second, 1)) // explicit
|
|
fa.New(fa.WithRequestsPerSecond(0.5)) // shorthand
|
|
```
|
|
|
|
## Iterators
|
|
|
|
Paginated endpoints return `iter.Seq2[*T, error]`:
|
|
|
|
```go
|
|
for sub, err := range client.Gallery(ctx, "someuser", fa.ListOptions{}) {
|
|
if err != nil { return err }
|
|
fmt.Println(sub.ID, sub.Title)
|
|
}
|
|
```
|
|
|
|
- **Lazy** pages fetched on demand; `break` stops further fetches.
|
|
- **Terminal errors** first error yields `(nil, err)` and stops the iterator.
|
|
- **One option-shape per iterator class**: simple paginated iterators
|
|
(Gallery / Scraps / Favorites / UserJournals / SubmissionInbox / Notes)
|
|
take `fa.ListOptions{StartPage, MaxPages}`; filtered iterators
|
|
(`Search`, `Browse`) take their own `SearchOptions` / `BrowseOptions`
|
|
struct that folds the same pagination fields in.
|
|
|
|
## Errors
|
|
|
|
Test with `errors.Is`:
|
|
|
|
| Sentinel | When | What to do |
|
|
|----------|------|-----------|
|
|
| `ErrNotFound` | FA "not found" / deleted submission | surface to user |
|
|
| `ErrUnauthorized` | endpoint needs login or `Login Required` page | refresh `a`/`b` |
|
|
| `ErrCloudflareChallenge` | CF interposed a challenge | refresh `cf_clearance` |
|
|
| `ErrRateLimited` | 429 after retry budget exhausted | slow down |
|
|
| `ErrSystemMessage` | uncategorised FA system message | inspect `*SystemMessageError` |
|
|
| `ErrParse` | HTML didn't match selectors | likely FA shifted markup open an issue |
|
|
|
|
## Test fixtures
|
|
|
|
Parsers are selector-driven; FA's HTML drifts. `TestRefreshFixtures` (build
|
|
tag `fixtures`) hits live FA with your cookies and snapshots each curated
|
|
page into `testdata/html/`. Parser `*_RealFixture` tests `t.Skip` cleanly
|
|
when their file is absent, so the basic suite works without any cookies.
|
|
|
|
```bash
|
|
# Configure once (or edit scripts/test.sh):
|
|
export FA_A=... FA_B=... CF_CLEARANCE=... FA_UA='Mozilla/5.0 ...'
|
|
export FA_TEST_USER=yourlogin FA_TEST_SUB_ID=... FA_TEST_JOURNAL_ID=...
|
|
# Optional refinements: FA_TEST_SUB_STORY_ID, FA_TEST_USER_WITH_SHOUTS,
|
|
# FA_TEST_GALLERY_USER, FA_TEST_GALLERY_LAST_PAGE, FA_TEST_NOTE_ID,
|
|
# FA_TEST_SEARCH_QUERY each gates a separate fixture; unset = skip.
|
|
|
|
go test -tags=fixtures -run TestRefreshFixtures -v ./... # or ./scripts/test.sh
|
|
go test ./... # now *_RealFixture tests activate
|
|
``` |