Mult Protocol
This commit is contained in:
@@ -386,25 +386,25 @@ front end never knows it is talking to a record protocol. It holds:
|
||||
* `downloadOffset` — next byte to request,
|
||||
* `consumedOffset` — next unread byte, sent as `ack`,
|
||||
* `readBuf` — received but not yet delivered to the reader,
|
||||
* two independent `requestLane`s, one for uploads and one for downloads, so a
|
||||
* two `requestLane`s of its own, one for uploads and one for downloads, so a
|
||||
blocking download poll never delays an upload.
|
||||
|
||||
`Write` slices the caller's buffer into records of the current upload size, each
|
||||
acknowledged before the next is sent. `Read` refills `readBuf` via
|
||||
`fillReadBuffer`, which issues batched download requests. `Close` sends
|
||||
`ModeClose` on a throwaway lane and closes both persistent lanes.
|
||||
`ModeClose` over the upload lane, then closes both lanes.
|
||||
|
||||
### 5.3 Request lanes and connection reuse
|
||||
|
||||
A `requestLane` owns at most one physical TCP connection and serialises requests
|
||||
onto it under a mutex. Any I/O error discards the connection; the next request
|
||||
redials. Sockets get `TCP_NODELAY`, 30-second keepalives, and optionally explicit
|
||||
buffer sizes via `--tcp-buffer`.
|
||||
onto it under a mutex. Each tunnel has two of them (§5.7). Any I/O error discards
|
||||
the connection; the next request redials. Sockets get `TCP_NODELAY`, 30-second
|
||||
keepalives, and optionally explicit buffer sizes via `--tcp-buffer`.
|
||||
|
||||
| `--chunk-reconnect-every` | Behaviour |
|
||||
|---|---|
|
||||
| `0` | Persistent — one connection for the life of the lane |
|
||||
| `1` | Auto — persistent if the path probe showed reuse works, otherwise one logical request per connection |
|
||||
| `1` (default) | Auto — persistent if the path probe showed reuse works, otherwise one logical request per connection. Resolved silently, since it runs once per flow |
|
||||
| `N ≥ 2` | Rotate — close and redial after N logical requests |
|
||||
|
||||
### 5.4 Path probing
|
||||
@@ -510,7 +510,33 @@ count = max(count, minPipeline) // the floor always wins
|
||||
The server independently clamps `count` to 256 and `limit` to its own
|
||||
`--chunk-max`, so a client can never demand more than the server allows.
|
||||
|
||||
### 5.7 Failure escalation
|
||||
### 5.7 Connection model
|
||||
|
||||
Each proxied socket gets its own tunnel, and each tunnel dials **two** TCP
|
||||
connections to the server: one upload lane and one download lane. `OPEN` and
|
||||
`CLOSE` ride the upload lane rather than dialling their own connections.
|
||||
|
||||
So a device browsing normally holds roughly `2 × active flows` connections to
|
||||
port 53, plus churn as flows come and go. That is the transport behaving as a
|
||||
proxy, not as a single multiplexed link.
|
||||
|
||||
**Why it is not one connection.** A response header is only `status + length` —
|
||||
it carries no session or request ID. Responses can therefore only be matched to
|
||||
requests by **arrival order**, which means a connection must finish one full
|
||||
exchange before another session may use it. Since a download is a long poll that
|
||||
can block for `--chunk-poll-wait`, sharing one connection across sessions lets
|
||||
idle pollers starve real traffic. Per-tunnel lanes are a requirement of the
|
||||
current wire format, not an oversight.
|
||||
|
||||
Making the client hold a single connection would require adding a session ID to
|
||||
the response header, demultiplexing responses asynchronously on the client, and
|
||||
handling requests concurrently per connection on the server — a wire-format
|
||||
change affecting both ends.
|
||||
|
||||
`--chunk-pollers` is accepted for compatibility and validated to 1–128, but the
|
||||
transport uses one download worker per tunnel and never reads it.
|
||||
|
||||
### 5.8 Failure escalation
|
||||
|
||||
On a download failure the client escalates in a fixed order:
|
||||
|
||||
@@ -800,15 +826,14 @@ checksum of zero is written as `0xFFFF` per RFC 768.
|
||||
| Min chunk | 32 | `--chunk-min` |
|
||||
| Batch max | 1 | `--chunk-concurrency` |
|
||||
| Batch min | 1 | `--chunk-concurrency-min` |
|
||||
| Reconnect every | 0 | `--chunk-reconnect-every` |
|
||||
| Reconnect every | 1 (auto) | `--chunk-reconnect-every` |
|
||||
| Timeout (s) | 2 | `--chunk-timeout` |
|
||||
|
||||
Fixed by the service: `--listen-host 127.0.0.1`, `--listen-port 8080`,
|
||||
`--transport chunk`, `--chunk-pollers 1`, `--chunk-grow-after 16`,
|
||||
`--chunk-adapt-log=true`.
|
||||
`--transport chunk`, `--chunk-grow-after 16`, `--chunk-adapt-log=true`.
|
||||
|
||||
Settings are laid out in four cards: CONNECTION, RECORD SIZE, DOWNLOAD BATCH, and
|
||||
ADVANCED. The batch card carries a live hint that restates the current setting in
|
||||
Settings are laid out in four cards: CONNECTION, RECORD SIZE, DOWNLOAD BATCH,
|
||||
and ADVANCED. The batch card carries a live hint that restates the current setting in
|
||||
words as you type, so the mode is never ambiguous:
|
||||
|
||||
```text
|
||||
@@ -827,7 +852,8 @@ Download batch: pinned at 5 records per request (never adapts)
|
||||
```
|
||||
|
||||
Validation ranges: port 1–65535, max chunk 32–1048576, min chunk 32–max chunk,
|
||||
batch values 1–256 with `min ≤ max`, reconnect 0–1000000, timeout 1–120.
|
||||
batch values 1–256 with `min ≤ max`, reconnect 0–1000000,
|
||||
timeout 1–120.
|
||||
|
||||
### 7.10 Logs
|
||||
|
||||
@@ -1013,7 +1039,8 @@ Max chunk: 1048576
|
||||
Min chunk: 32
|
||||
Batch max: 1
|
||||
Batch min: 1
|
||||
Reconnect every: 0
|
||||
Pollers: 1
|
||||
Reconnect every: 1
|
||||
Timeout (s): 2
|
||||
```
|
||||
|
||||
@@ -1065,7 +1092,7 @@ Use **OPEN LOGS** to watch the path probe and any adaptation.
|
||||
| `--chunk-reconnect-every` | `0` | 0 persistent, 1 auto, N rotate |
|
||||
| `--chunk-poll-delay` | `2ms` | Pause after an empty poll |
|
||||
| `--chunk-timeout` | `2s` | Per-record transaction timeout |
|
||||
| `--chunk-pollers` | `1` | Reserved compatibility knob; accepted but unused |
|
||||
| `--chunk-pollers` | `1` | Accepted for compatibility; validated 1–128 but unused |
|
||||
|
||||
The `concurrency` flag names are historical. They control the download **batch
|
||||
depth** described in §5.6, not any form of threading.
|
||||
@@ -1099,6 +1126,15 @@ manual tuning makes things worse.
|
||||
1. More records per request is the main lever when latency to the server is
|
||||
high, because each round trip returns more data. If the log repeatedly shows
|
||||
`adaptive download batch: N -> N/2`, the path cannot sustain that depth.
|
||||
* **Streams die mid-transfer, or nothing loads at all.** Set `Reconnect every`
|
||||
to `1` (auto). `0` forces persistent connections, and many networks silently
|
||||
kill long-lived port-53 connections; auto probes first and falls back to one
|
||||
logical request per connection when persistence does not survive. This is the
|
||||
single most important setting on a restrictive path.
|
||||
* **Logs show the same transition many times over (`128 -> 64` repeatedly).**
|
||||
Each tunnel adapts independently, so a burst of flows produces a burst of
|
||||
identical lines. The app collapses consecutive duplicates into a counted line;
|
||||
the underlying behaviour is normal.
|
||||
* **The batch keeps collapsing to 1 and throughput dies with it.** Some paths only
|
||||
deliver correctly at one specific number of records. Try `Batch max = Batch min
|
||||
= N` for a few values of N and leave it pinned at whichever works. Pinned mode
|
||||
|
||||
Reference in New Issue
Block a user