diff --git a/README.md b/README.md
index 746ac98..5bf5f5f 100644
--- a/README.md
+++ b/README.md
@@ -90,10 +90,11 @@ flow control a matter of reporting a number.
**DragonTCP does not provide authenticated encryption. It is not a VPN in the
security sense.**
-Payloads are **masked**: XORed with a keystream derived from SHA-256 that varies
-with session ID, mode, sequence, direction, and block number. The same plaintext
-therefore does not produce the same ciphertext twice, and no fixed ASCII markers
-appear on the wire.
+Legacy payloads are **masked**: XORed with a keystream derived from SHA-256 that
+varies with session ID, mode, sequence, direction, and block number. New clients
+also support a self-described **clear-payload** profile that removes the per-32-byte
+SHA-256 work. Auto tries clear first and falls back to the legacy mask for older
+servers or networks that reject the clear profile.
This defeats trivial pattern matching. It does not defeat anyone who can read the
traffic:
@@ -101,7 +102,9 @@ traffic:
* The mask is derived from the **session ID, which is transmitted in cleartext in
every request header.** Anyone who sees the header can regenerate the keystream
and recover the plaintext. This is obfuscation, not confidentiality.
-* Record headers — mode, session, sequence, length — are never masked.
+* Session, sequence, and length fields remain clear. The mode/status byte may
+ use a startup-selected header mask, but this is traffic shaping rather than
+ cryptographic protection.
* `StatusError` bodies are sent **unmasked**, as plain text.
* There is no integrity check, so payloads can be tampered with undetected.
@@ -130,6 +133,7 @@ core/
cmd/dragontcp-server/
main.go listener, DNS cache, address filtering, CLI flags
chunk.go session manager, buffering, request dispatch
+ bhttp.go auto-detected BP/BHP1 transport compatibility
debug.go counters and periodic statistics
chunk_test.go
internal/wire/
@@ -227,7 +231,7 @@ A single request may be answered by **several** responses — see `ModeDownload`
| `StatusWait` | 3 | Nothing available yet; ask again |
| `StatusEOF` | 4 | Target closed the stream |
-### 4.3 Payload masking
+### 4.3 Payload encoding
```text
seed[0:16] = session ID
@@ -245,6 +249,12 @@ sequence field is a **byte offset** (§4.5), consecutive records never reuse a
keystream position, and re-sending the same offset reproduces the same bytes —
which is what makes idempotent retries safe.
+When the cover preface carries the clear-payload flag, request and data bodies are
+sent without that transform. Headers, session semantics, retry offsets, and all
+payload layouts remain identical. Old clients remain masked and are accepted by
+the new server. An old server rejects the new flag, so the new client's next
+startup candidate is the corresponding legacy direct profile.
+
Not everything is masked. `WriteResponse` sends the body as-is and is used for
empty `StatusOK`, `StatusWait`, `StatusEOF`, and every `StatusError`.
`WriteMaskedResponse` is used for `StatusData` and for the `OPEN` result. On the
@@ -350,6 +360,27 @@ client server
│◀─ OK ────────────────────────────────────────│
```
+### 4.7 BP transport compatibility
+
+Binary connections are auto-detected as either native Dragon B or the
+observable BP protocol used by `bhttp_remote_test.py`. BP support uses the
+same 29-byte request header, five-byte response header, and SHA-256 counter mask,
+but maps modes as `0=probe`, `1=upload/register`, `2=single download`, `3=batch
+download`, and `4=ACK`. It implements `BHP1` version-1 probe integrity, probe
+batching, empty mode-1 session registration, upload acknowledgements, mode-2's
+header-only size hint, the six-byte batch request, ACK, expiry, and unknown-session
+errors. Dragon peers can additionally negotiate the clear-payload encoding via
+the cover preface; reference clients continue through the original direct masked
+encoding unchanged.
+
+The available reference client does not expose a destination-selection or
+authentication exchange. Its observable registration/upload/download/ACK
+behavior remains accepted unchanged. Dragon's BP client adds a `DOP1` upload
+extension after registration to carry the normal token, target host, and port;
+that extension gives the Android app a complete bidirectional stream without
+changing the reference client's frames. Native Dragon B remains available and
+is not replaced.
+
---
## 5. The Go client
@@ -404,10 +435,44 @@ keepalives, and optionally explicit buffer sizes via `--tcp-buffer`.
| `--chunk-reconnect-every` | Behaviour |
|---|---|
| `0` | Persistent — one connection for the life of the lane |
-| `1` (default) | Auto — persistent if the path probe showed reuse works, otherwise one logical request per connection. Resolved silently, since it runs once per flow |
+| `1` (app default) | Auto — starts persistent, retries once after reconnect, and switches that lane to one request per connection only when reuse fails during real traffic |
| `N ≥ 2` | Rotate — close and redial after N logical requests |
-### 5.4 Path probing
+### 5.4 Startup profile and path probing
+
+Before normal traffic, the client discovers one fixed wire/header profile. The
+original direct range remains intact: B provides 32 masks for its mode byte, X
+provides 96 masks for its `UP`/`OK` magic, and BP provides its original direct
+profile. The cover range adds both legacy-masked and clear-payload B profiles,
+clear-payload BP profiles, and covered X profiles. It varies a masked multi-byte
+preface, all 256 header-mask/first-byte values, and a distributed set of padding
+lengths: `0, 1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 768, 1024, 1400, 2048,
+4096` bytes.
+
+Every candidate is tested with real tunnel traffic: the client opens the tunnel,
+sends an HTTP request to `http://ip.dr2.site/` over TCP port 80, and accepts the
+profile only after receiving an HTTP status line. Discovery no longer rejects a
+profile based only on the synthetic `CPROBE`/`DTP2` exchange.
+
+The safe default is one probe thread. `--wire-probe-threads` can allow 1–16
+in-flight attempts, while `--wire-probe-delay` (default 1 second) is still the
+global minimum time between new connection starts. Increasing the thread count
+therefore permits slow attempts to overlap; it does not launch all candidates at
+once. One thread is recommended on carriers with connection-rate filtering.
+
+The winner is cached for the lifetime of the client process and is used
+unchanged for every connection and record. Padding contents may be fresh random
+bytes, but the selected length and profile never change until restart; there is
+no per-packet profile mutation.
+
+`--wire auto` searches 1,153 distributed B/BP/X profiles. Pinning `--wire b`
+searches 544 B profiles, `--wire bp` searches 257 BP profiles, and `--wire x`
+searches 352 X profiles. B and BP try a clear-payload candidate first, followed
+immediately by a legacy direct fallback. A successful selection is logged as:
+
+```text
+wire probe: selected=x/mask-6b/cover-91e7/pad-64 completed=141 launched=142 elapsed=42.8s target=http://ip.dr2.site/ validated=true threads=1 fixed_until_restart=true
+```
Before the first real connection, `getPathProfile` measures the path once and
caches the result for **30 minutes**, keyed by server address, token, and size
@@ -558,10 +623,11 @@ Source: `core/cmd/dragontcp-server/`.
### 6.1 Connection handling
The server listens on `0.0.0.0:53` by default, bounded by `--max-connections`
-through a slot channel. Each connection runs a loop: read one request with a
-30-second deadline, dispatch it, repeat. Because session state is keyed by session
-ID rather than by connection, requests for one logical stream may arrive over many
-connections in whatever pattern the client chooses.
+through a slot channel. Each connection runs a loop: read one request with an
+idle deadline, dispatch it, repeat. The deadline is refreshed halfway through its
+window rather than issuing a system call for every record. Because session state
+is keyed by session ID rather than by connection, requests for one logical stream
+may arrive over many connections in whatever pattern the client chooses.
### 6.2 Session state and buffering
@@ -574,7 +640,8 @@ target plus a download buffer:
That goroutine **blocks when the buffer is full**, which is the entire flow
control story described in §4.5. Buffer size is `--chunk-buffered × 65536`,
-clamped to 1 MiB…64 MiB (default 256 → 16 MiB per session).
+clamped to 1 MiB…64 MiB (default 32 → 2 MiB per session). The same byte limit is
+enforced for X; it is not multiplied by the negotiated chunk size.
Acknowledgement drops bytes off the front and advances `base`. The buffer is
compacted when its capacity exceeds four times its length and is over 1 MiB, so
@@ -673,9 +740,9 @@ a helper binary in an APK.
`DragonService` launches it with `ProcessBuilder`, merges stderr into stdout, and
reads the output on a background thread. Only interesting lines reach the UI log:
-those beginning with `adaptive ` or `path probe:`, and anything containing `error`
-or `failed`. A watchdog thread waits on the process; if the core exits while the
-tunnel is supposed to be up, the whole VPN is torn down.
+those beginning with `wire`, `adaptive `, or `path probe:`, and anything
+containing `error` or `failed`. A watchdog thread waits on the process; if the
+core exits while the tunnel is supposed to be up, the whole VPN is torn down.
### 7.3 Startup and shutdown
@@ -822,12 +889,15 @@ checksum of zero is written as `0xFFFF` per RFC 768.
| Server | — | `--server-host` |
| Port | 53 | `--server-port` |
| Token | empty | `--token` (omitted entirely when blank) |
+| Wire | auto | `--wire`; Auto, B, BP, and X are selectable |
+| Probe delay (ms) | 1000 | `--wire-probe-delay`; global minimum delay between profile connection starts |
+| Probe threads | 1 | `--wire-probe-threads`; maximum concurrent profile attempts, 1–16 |
| Max chunk | 1048576 | `--chunk-max` **and** `--chunk-start` |
| Min chunk | 32 | `--chunk-min` |
| Batch max | 1 | `--chunk-concurrency` |
| Batch min | 1 | `--chunk-concurrency-min` |
| Reconnect every | 1 (auto) | `--chunk-reconnect-every` |
-| Timeout (s) | 2 | `--chunk-timeout` |
+| Timeout (s) | 5 | `--chunk-timeout` |
Fixed by the service: `--listen-host 127.0.0.1`, `--listen-port 8080`,
`--transport chunk`, `--chunk-grow-after 16`, `--chunk-adapt-log=true`.
@@ -853,7 +923,7 @@ Download batch: pinned at 5 records per request (never adapts)
Validation ranges: port 1–65535, max chunk 32–1048576, min chunk 32–max chunk,
batch values 1–256 with `min ≤ max`, reconnect 0–1000000,
-timeout 1–120.
+timeout 1–120, probe delay 200–30000 ms, probe threads 1–16.
### 7.10 Logs
@@ -989,26 +1059,30 @@ keystore out of version control.
### 9.1 Server
```bash
-sudo ./dragontcp-hybrid-server-linux-amd64 --port 53 --chunk-max 1048576
+sudo ./dragontcp-hybrid-server-linux-amd64 \
+ --port 53 --port-alt 80 --chunk-max 1048576
```
With a token:
```bash
sudo ./dragontcp-hybrid-server-linux-amd64 \
- --token 'YOUR_SECRET' --port 53 --chunk-max 1048576
+ --token 'YOUR_SECRET' --port 53 --port-alt 80 --chunk-max 1048576
```
With diagnostics:
```bash
sudo ./dragontcp-hybrid-server-linux-amd64 \
- --port 53 --chunk-max 1048576 --debug --debug-stats-interval 10s
+ --port 53 --port-alt 80 --chunk-max 1048576 --debug --debug-stats-interval 10s
```
-`sudo` is needed only because port 53 is privileged. If `systemd-resolved` or
-`dnsmasq` already owns port 53, free it or choose another port. No TUN device, NAT,
-or firewall rules are required.
+The server listens on TCP ports 53 and 80 simultaneously by default. Set
+`--port-alt 0` to disable the second listener. Failure to bind the primary port
+is fatal; failure to bind the secondary port prints a warning and leaves the
+primary listener running. `sudo` is normally needed because both defaults are
+privileged. If another service owns either port, free it or select a different
+port. No TUN device or NAT rules are required.
### 9.2 Client CLI
@@ -1023,7 +1097,7 @@ summarises the active configuration:
```text
adaptive_chunk=true start=1048576 min=32 max=1048576 grow_after=16 pollers=1 \
-batch=5-5(pinned) reconnect_every=0 timeout=2s
+batch=5-5(pinned) reconnect_every=0 timeout=5s
```
### 9.3 Android
@@ -1041,7 +1115,9 @@ Batch max: 1
Batch min: 1
Pollers: 1
Reconnect every: 1
-Timeout (s): 2
+Timeout (s): 5
+Probe delay (ms): 1000
+Probe threads: 1
```
Use **OPEN LOGS** to watch the path probe and any adaptation.
@@ -1055,7 +1131,8 @@ Use **OPEN LOGS** to watch the path probe and any adaptation.
| Flag | Default | Meaning |
|---|---|---|
| `--host` | `0.0.0.0` | Listen address |
-| `--port` | `53` | Listen port |
+| `--port` | `53` | Primary listen port |
+| `--port-alt` | `80` | Simultaneous secondary listen port; 0 disables it |
| `--token` | empty | Optional shared secret |
| `--max-connections` | `20000` | Concurrent TCP connections |
| `--allow-private` | `false` | Allow private/loopback targets — keep off in public |
@@ -1063,7 +1140,7 @@ Use **OPEN LOGS** to watch the path probe and any adaptation.
| `--dns-cache-size` | `4096` | Cached hostnames |
| `--tcp-buffer` | `0` | Explicit socket buffers; 0 = OS autotuning |
| `--chunk-max` | `1048576` | Largest record accepted (32 B – 1 MiB) |
-| `--chunk-buffered` | `256` | Per-session buffer in 64 KiB units (≈16 MiB) |
+| `--chunk-buffered` | `32` | Per-session buffer in 64 KiB units (≈2 MiB) |
| `--chunk-poll-wait` | `200ms` | Long-poll wait for a batch's first record |
| `--chunk-session-timeout` | `2m` | Idle session reaping |
| `--debug` | `false` | Session/connect/error logs plus periodic stats |
@@ -1078,6 +1155,9 @@ Use **OPEN LOGS** to watch the path probe and any adaptation.
| `--server-host` / `--server-port` | — / `53` | Remote server (host required) |
| `--token` | empty | Shared secret |
| `--transport` | `chunk` | Must be `chunk` |
+| `--wire` | `auto` | Search B/BP/X; `b`, `bp`, or `x` pins one mode |
+| `--wire-probe-delay` | `1s` | Global minimum delay between profile probe starts; range 200ms–30s |
+| `--wire-probe-threads` | `1` | Maximum concurrent real HTTP profile probes; range 1–16 |
| `--max-connections` | `20000` | Concurrent proxied connections |
| `--tcp-buffer` | `0` | Explicit socket buffers |
| `--chunk-start` | `1048576` | Initial record size (the probe overrides it) |
@@ -1091,7 +1171,7 @@ Use **OPEN LOGS** to watch the path probe and any adaptation.
| `--chunk-concurrency-min` | `1` | Download batch floor, 1–256; equal to the ceiling pins the depth |
| `--chunk-reconnect-every` | `0` | 0 persistent, 1 auto, N rotate |
| `--chunk-poll-delay` | `2ms` | Pause after an empty poll |
-| `--chunk-timeout` | `2s` | Per-record transaction timeout |
+| `--chunk-timeout` | `5s` | Per-record transaction timeout |
| `--chunk-pollers` | `1` | Accepted for compatibility; validated 1–128 but unused |
The `concurrency` flag names are historical. They control the download **batch
@@ -1127,10 +1207,10 @@ manual tuning makes things worse.
high, because each round trip returns more data. If the log repeatedly shows
`adaptive download batch: N -> N/2`, the path cannot sustain that depth.
* **Streams die mid-transfer, or nothing loads at all.** Set `Reconnect every`
- to `1` (auto). `0` forces persistent connections, and many networks silently
- kill long-lived port-53 connections; auto probes first and falls back to one
- logical request per connection when persistence does not survive. This is the
- single most important setting on a restrictive path.
+ to `1` (auto). Auto starts persistent and retries an idempotent request once on
+ a fresh connection. If reuse itself failed, only that lane switches to one
+ logical request per connection. This avoids turning one imperfect startup
+ probe into connection churn for every active tunnel.
* **Logs show the same transition many times over (`128 -> 64` repeatedly).**
Each tunnel adapts independently, so a burst of flows produces a burst of
identical lines. The app collapses consecutive duplicates into a counted line;
@@ -1151,7 +1231,8 @@ manual tuning makes things worse.
a small number of high-BDP connections is `1048576` or `4194304` worth trying;
across many connections it costs memory for nothing.
* **Server memory.** Each session can hold `--chunk-buffered × 64 KiB` (default
- 16 MiB). Lower it when running many concurrent sessions.
+ 2 MiB). Lower it when running many concurrent sessions; the minimum effective
+ window is 1 MiB.
---
@@ -1196,8 +1277,8 @@ cd core && go test ./...
Coverage:
-* the masking round-trip, and that different sequences produce different wire
- bytes,
+* legacy masking round-trips and different-sequence wire bytes,
+* clear-payload B and BP profiles end to end, including nonzero header masks,
* the adaptive record sizer recovering from the minimum rather than latching
there,
* `reconnectEvery == 0` meaning persistent,
diff --git a/SHA256SUMS b/SHA256SUMS
index 753a4c0..0d6fb03 100644
--- a/SHA256SUMS
+++ b/SHA256SUMS
@@ -1,6 +1,6 @@
-56707362bae6b388795150a77b27a14de046a56e05037485d5dfb4bb7db0f8b9 *bin/dragontcp-hybrid-server-linux-amd64
-35e7dbbb84bbb76b0eea052eff18d58e24c1fa1c43274aa6c1a5c686a1d38f40 *bin/dragontcp-hybrid-server-linux-arm64
-2f820ce82a65c285684c875afd0311f1d62afe856f8b878eab682f9864252a19 *bin/dragontcp-hybrid-client-linux-amd64
-1b5bcf4d3a446cec397206557fa6f1a4c229b11b8e871a7a1072ac3ae9f81e31 *android/lib/arm64-v8a/libdragontcp_client.so
-8021680a9cff84b0dcdec3c6a1f38c5b6f9620aba2bed89766920e4b2f514afa *android/lib/armeabi-v7a/libdragontcp_client.so
-baa7e4e304f96edd4d1fe8f22b9f11398e1d8ad00e09454afca8419ae718bdd2 *android/lib/x86_64/libdragontcp_client.so
+969e4a020d5475bfff355f38ca809edb68686b5850af8f9709126ed5bb2d13bb *bin/dragontcp-hybrid-server-linux-amd64
+eaf2d07d8edf17840ce7bad4b1c81e8f3ccb1504c18397a773ca0aacb2ee72ef *bin/dragontcp-hybrid-server-linux-arm64
+b6dda0c38b8472b8e648a212daddca5a737c8e0a3457eb3c4f29833ec3d2e1c4 *bin/dragontcp-hybrid-client-linux-amd64
+d70e5cb83cb6109cba2a8dab1b232c5b7545abb26bb9ad71b92972b7e8f830d9 *android/lib/arm64-v8a/libdragontcp_client.so
+3c45f678da399ecd20c28b8acc631d8c6f18151415ac63109f838a8c68a00e68 *android/lib/armeabi-v7a/libdragontcp_client.so
+78806799b1c2949835e854635f6c352b572d3b3b526c4a6de877da9dd4e2d98b *android/lib/x86_64/libdragontcp_client.so
diff --git a/android/.idea/deviceManager.xml b/android/.idea/deviceManager.xml
new file mode 100644
index 0000000..91f9558
--- /dev/null
+++ b/android/.idea/deviceManager.xml
@@ -0,0 +1,13 @@
+
+
+
+
+
+
\ No newline at end of file
diff --git a/android/lib/arm64-v8a/libdragontcp_client.so b/android/lib/arm64-v8a/libdragontcp_client.so
index b69a4d9..255db71 100644
Binary files a/android/lib/arm64-v8a/libdragontcp_client.so and b/android/lib/arm64-v8a/libdragontcp_client.so differ
diff --git a/android/lib/armeabi-v7a/libdragontcp_client.so b/android/lib/armeabi-v7a/libdragontcp_client.so
index 370d4c1..904fdd8 100644
Binary files a/android/lib/armeabi-v7a/libdragontcp_client.so and b/android/lib/armeabi-v7a/libdragontcp_client.so differ
diff --git a/android/lib/x86_64/libdragontcp_client.so b/android/lib/x86_64/libdragontcp_client.so
index 5452868..7480431 100644
Binary files a/android/lib/x86_64/libdragontcp_client.so and b/android/lib/x86_64/libdragontcp_client.so differ
diff --git a/android/src/com/dragontcp/client/DragonService.java b/android/src/com/dragontcp/client/DragonService.java
index 6396ea7..c2a87cf 100644
--- a/android/src/com/dragontcp/client/DragonService.java
+++ b/android/src/com/dragontcp/client/DragonService.java
@@ -38,7 +38,7 @@ public class DragonService extends VpnService {
public static final String EXTRA_SERVER = "server";
public static final String EXTRA_PORT = "port";
public static final String EXTRA_TOKEN = "token";
- /** Wire format: "auto", "b" or "x". */
+ /** Wire format: "auto", "b", "bp" or "x". */
public static final String EXTRA_WIRE = "wire";
public static final String EXTRA_CHUNK_MAX = "chunkMax";
public static final String EXTRA_CHUNK_MIN = "chunkMin";
@@ -48,6 +48,8 @@ public class DragonService extends VpnService {
public static final String EXTRA_BATCH_MIN = "batchMin";
public static final String EXTRA_RECONNECT = "reconnect";
public static final String EXTRA_TIMEOUT = "timeout";
+ public static final String EXTRA_PROBE_DELAY = "probeDelay";
+ public static final String EXTRA_PROBE_THREADS = "probeThreads";
private static final int BATCH_LIMIT = 256;
@@ -118,7 +120,9 @@ public class DragonService extends VpnService {
int batchMax = intent.getIntExtra(EXTRA_BATCH_MAX, 1);
int batchMin = intent.getIntExtra(EXTRA_BATCH_MIN, 1);
int reconnect = intent.getIntExtra(EXTRA_RECONNECT, 1);
- int timeout = intent.getIntExtra(EXTRA_TIMEOUT, 2);
+ int timeout = intent.getIntExtra(EXTRA_TIMEOUT, 5);
+ int probeDelay = intent.getIntExtra(EXTRA_PROBE_DELAY, 1000);
+ int probeThreads = intent.getIntExtra(EXTRA_PROBE_THREADS, 1);
if (server == null || server.trim().isEmpty()) {
failStart("Server is required");
@@ -126,19 +130,22 @@ public class DragonService extends VpnService {
}
server = server.trim();
if (token == null) token = "";
- if (wire == null || !(wire.equals("b") || wire.equals("x"))) wire = "auto";
+ if (wire == null || !(wire.equals("b") || wire.equals("bp") || wire.equals("x"))) wire = "auto";
chunkMax = Math.max(32, Math.min(1024 * 1024, chunkMax));
chunkMin = Math.max(32, Math.min(chunkMax, chunkMin));
batchMax = Math.max(1, Math.min(BATCH_LIMIT, batchMax));
batchMin = Math.max(1, Math.min(batchMax, batchMin));
reconnect = Math.max(0, reconnect);
timeout = Math.max(1, timeout);
+ probeDelay = Math.max(200, Math.min(30000, probeDelay));
+ probeThreads = Math.max(1, Math.min(16, probeThreads));
try {
AppLog.append("Starting DragonTCP → " + server + ":" + port);
AppLog.append(describeBatch(batchMin, batchMax));
AppLog.append(describeWire(wire));
- Process process = startDragonCore(server, port, token, chunkMax, chunkMin, batchMax, batchMin, wire, reconnect, timeout);
+ AppLog.append("Wire probe: http://ip.dr2.site/ • delay " + probeDelay + " ms • threads " + probeThreads);
+ Process process = startDragonCore(server, port, token, chunkMax, chunkMin, batchMax, batchMin, wire, reconnect, timeout, probeDelay, probeThreads);
synchronized (stateLock) { coreProcess = process; }
startCoreLogReader(process);
@@ -206,7 +213,9 @@ public class DragonService extends VpnService {
int batchMin,
String wire,
int reconnect,
- int timeout
+ int timeout,
+ int probeDelay,
+ int probeThreads
) throws Exception {
File executable = new File(getApplicationInfo().nativeLibraryDir, "libdragontcp_client.so");
if (!executable.exists()) throw new IllegalStateException("Embedded DragonTCP core is missing");
@@ -224,6 +233,8 @@ public class DragonService extends VpnService {
cmd.add("--chunk-max"); cmd.add(Integer.toString(chunkMax));
cmd.add("--chunk-pollers"); cmd.add("1");
cmd.add("--wire"); cmd.add(wire);
+ cmd.add("--wire-probe-delay"); cmd.add(probeDelay + "ms");
+ cmd.add("--wire-probe-threads"); cmd.add(Integer.toString(probeThreads));
cmd.add("--chunk-concurrency"); cmd.add(Integer.toString(batchMax));
cmd.add("--chunk-concurrency-min"); cmd.add(Integer.toString(batchMin));
cmd.add("--chunk-reconnect-every"); cmd.add(Integer.toString(reconnect));
@@ -249,9 +260,10 @@ public class DragonService extends VpnService {
/** Human-readable summary of the wire selection, for the log screen. */
private static String describeWire(String wire) {
- if ("b".equals(wire)) return "Wire: B (manual)";
- if ("x".equals(wire)) return "Wire: X (manual)";
- return "Wire: auto (probing)";
+ if ("b".equals(wire)) return "Wire: B (discovering header profile)";
+ if ("bp".equals(wire)) return "Wire: BP";
+ if ("x".equals(wire)) return "Wire: X (discovering header profile)";
+ return "Wire: auto (discovering B/BP/X profile)";
}
private void startCoreLogReader(Process process) {
diff --git a/android/src/com/dragontcp/client/MainActivity.java b/android/src/com/dragontcp/client/MainActivity.java
index 96f6b69..625ab9a 100644
--- a/android/src/com/dragontcp/client/MainActivity.java
+++ b/android/src/com/dragontcp/client/MainActivity.java
@@ -55,8 +55,11 @@ public class MainActivity extends Activity {
private EditText batchMax;
private EditText batchMin;
private EditText reconnect;
+ private EditText probeDelay;
+ private EditText probeThreads;
private Button wireAuto;
private Button wireB;
+ private Button wireBP;
private Button wireX;
private String wireMode = "auto";
private EditText timeout;
@@ -225,9 +228,11 @@ public class MainActivity extends Activity {
LinearLayout wireRow = row();
wireAuto = segmentButton("A");
wireB = segmentButton("B");
+ wireBP = segmentButton("BP");
wireX = segmentButton("X");
addSegment(wireRow, wireAuto);
addSegment(wireRow, wireB);
+ addSegment(wireRow, wireBP);
addSegment(wireRow, wireX);
wireCard.addView(wireRow);
wireHint = text("", 11, MUTED, true);
@@ -238,6 +243,7 @@ public class MainActivity extends Activity {
wireAuto.setOnClickListener(v -> setWireMode("auto"));
wireB.setOnClickListener(v -> setWireMode("b"));
+ wireBP.setOnClickListener(v -> setWireMode("bp"));
wireX.setOnClickListener(v -> setWireMode("x"));
// --------------------------------------------------------- record size
@@ -282,11 +288,17 @@ public class MainActivity extends Activity {
LinearLayout advancedCard = card("ADVANCED");
LinearLayout timing = row();
reconnect = addFieldToRow(timing, "Reconnect every", "1 = auto", "1", true, false, 0.58f);
- timeout = addFieldToRow(timing, "Timeout (s)", "2", "2", true, false, 0.42f);
+ timeout = addFieldToRow(timing, "Timeout (s)", "5", "5", true, false, 0.42f);
advancedCard.addView(timing);
+ LinearLayout probing = row();
+ probeDelay = addFieldToRow(probing, "Probe delay (ms)", "1000", "1000", true, false, 0.65f);
+ probeThreads = addFieldToRow(probing, "Probe threads", "1", "1", true, false, 0.35f);
+ advancedCard.addView(probing);
advancedCard.addView(hint(
- "1 = auto (recommended: probes the path, falls back to one request per "
- + "connection) • 0 = persistent • N = rotate every N requests"));
+ "1 = auto (recommended: starts persistent, then learns one request per "
+ + "connection only if reuse really fails) • 0 = persistent • N = rotate every N requests\n"
+ + "Probe delay spaces startup profile connections; 1000 ms is recommended. "
+ + "Probe threads defaults to 1; more may trigger carrier limits."));
settings.addView(advancedCard, cardParams());
// ------------------------------------------------------------- buttons
@@ -469,17 +481,21 @@ public class MainActivity extends Activity {
wireMode = mode;
paintSegment(wireAuto, "auto".equals(mode));
paintSegment(wireB, "b".equals(mode));
+ paintSegment(wireBP, "bp".equals(mode));
paintSegment(wireX, "x".equals(mode));
if (wireHint == null) return;
if ("auto".equals(mode)) {
wireHint.setTextColor(ACCENT);
- wireHint.setText("Auto: tries B, then X, keeping the one that connects.");
+ wireHint.setText("Auto: discovers a working B/BP/X profile and keeps it until restart.");
} else if ("b".equals(mode)) {
wireHint.setTextColor(OK);
- wireHint.setText("Manual: B.");
+ wireHint.setText("B only; header profile is discovered automatically.");
+ } else if ("bp".equals(mode)) {
+ wireHint.setTextColor(OK);
+ wireHint.setText("BP only.");
} else {
wireHint.setTextColor(OK);
- wireHint.setText("Manual: X.");
+ wireHint.setText("X only; header profile is discovered automatically.");
}
}
@@ -578,7 +594,9 @@ public class MainActivity extends Activity {
i.putExtra(DragonService.EXTRA_BATCH_MAX, p.getInt("batchMax", 1));
i.putExtra(DragonService.EXTRA_BATCH_MIN, p.getInt("batchMin", 1));
i.putExtra(DragonService.EXTRA_RECONNECT, p.getInt("reconnect", 1));
- i.putExtra(DragonService.EXTRA_TIMEOUT, p.getInt("timeout", 2));
+ i.putExtra(DragonService.EXTRA_TIMEOUT, p.getInt("timeout", 5));
+ i.putExtra(DragonService.EXTRA_PROBE_DELAY, p.getInt("probeDelay", 1000));
+ i.putExtra(DragonService.EXTRA_PROBE_THREADS, p.getInt("probeThreads", 1));
if (Build.VERSION.SDK_INT >= 26) startForegroundService(i); else startService(i);
}
@@ -593,6 +611,8 @@ public class MainActivity extends Activity {
if (bMin > bMax) throw new IllegalArgumentException("Batch min must not exceed batch max");
int rec = parse(reconnect, 0, 1000000, "Reconnect every");
int tout = parse(timeout, 1, 120, "Timeout");
+ int delay = parse(probeDelay, 200, 30000, "Probe delay");
+ int threads = parse(probeThreads, 1, 16, "Probe threads");
getSharedPreferences(PREFS, MODE_PRIVATE).edit()
.putString("server", h)
@@ -605,6 +625,8 @@ public class MainActivity extends Activity {
.putInt("batchMin", bMin)
.putInt("reconnect", rec)
.putInt("timeout", tout)
+ .putInt("probeDelay", delay)
+ .putInt("probeThreads", threads)
.apply();
}
@@ -627,7 +649,9 @@ public class MainActivity extends Activity {
batchMax.setText(Integer.toString(p.getInt("batchMax", 1)));
batchMin.setText(Integer.toString(p.getInt("batchMin", 1)));
reconnect.setText(Integer.toString(p.getInt("reconnect", 1)));
- timeout.setText(Integer.toString(p.getInt("timeout", 2)));
+ timeout.setText(Integer.toString(p.getInt("timeout", 5)));
+ probeDelay.setText(Integer.toString(p.getInt("probeDelay", 1000)));
+ probeThreads.setText(Integer.toString(p.getInt("probeThreads", 1)));
}
private void updateConnectionUi(boolean active, String rawStatus) {
diff --git a/bin/dragontcp-hybrid-client-linux-amd64 b/bin/dragontcp-hybrid-client-linux-amd64
index 26dfe89..7e43009 100644
Binary files a/bin/dragontcp-hybrid-client-linux-amd64 and b/bin/dragontcp-hybrid-client-linux-amd64 differ
diff --git a/bin/dragontcp-hybrid-server-linux-amd64 b/bin/dragontcp-hybrid-server-linux-amd64
index f05687a..d7dcf88 100644
Binary files a/bin/dragontcp-hybrid-server-linux-amd64 and b/bin/dragontcp-hybrid-server-linux-amd64 differ
diff --git a/bin/dragontcp-hybrid-server-linux-arm64 b/bin/dragontcp-hybrid-server-linux-arm64
index f0cdb32..c1ed1dd 100644
Binary files a/bin/dragontcp-hybrid-server-linux-arm64 and b/bin/dragontcp-hybrid-server-linux-arm64 differ
diff --git a/bp_port_test.py b/bp_port_test.py
new file mode 100644
index 0000000..5ff6b95
--- /dev/null
+++ b/bp_port_test.py
@@ -0,0 +1,219 @@
+#!/usr/bin/env python3
+"""Test which TCP ports on an authorized host respond to the DragonTCP BP probe."""
+
+from __future__ import annotations
+
+import argparse
+import concurrent.futures
+import csv
+import hashlib
+import socket
+import struct
+import sys
+import threading
+import time
+import uuid
+from dataclasses import dataclass
+from pathlib import Path
+
+
+MODE_PROBE = 0
+STATUS_OK = 0
+REQUEST_HEADER_SIZE = 29
+RESPONSE_HEADER_SIZE = 5
+MAX_RESPONSE_BODY = 2 * 1024 * 1024
+BP_PROBE = b"BHP1\x01\x00\x00\x00\x00\x00"
+
+
+@dataclass(frozen=True)
+class Result:
+ port: int
+ state: str
+ elapsed_ms: int
+ detail: str = ""
+
+
+def sha256_ctr_mask(
+ data: bytes,
+ session_id: bytes,
+ mode: int,
+ sequence: int,
+ is_response: bool,
+) -> bytes:
+ """Apply the BP payload mask. Calling this twice restores the input."""
+ if not data:
+ return b""
+ sid = session_id[:16].ljust(16, b"\x00")
+ seed = sid + bytes((mode & 0xFF,)) + struct.pack(">Q", sequence)
+ seed += bytes((1 if is_response else 0,))
+ out = bytearray(len(data))
+ for offset in range(0, len(data), 32):
+ counter = offset // 32
+ block = hashlib.sha256(seed + struct.pack(">I", counter)).digest()
+ count = min(32, len(data) - offset)
+ for index in range(count):
+ out[offset + index] = data[offset + index] ^ block[index]
+ return bytes(out)
+
+
+def read_exact(sock: socket.socket, size: int) -> bytes:
+ data = bytearray()
+ while len(data) < size:
+ chunk = sock.recv(size - len(data))
+ if not chunk:
+ raise EOFError(f"EOF after {len(data)}/{size} bytes")
+ data.extend(chunk)
+ return bytes(data)
+
+
+def test_port(host: str, port: int, timeout: float) -> Result:
+ started = time.monotonic()
+ session_id = uuid.uuid4().bytes
+ encrypted = sha256_ctr_mask(BP_PROBE, session_id, MODE_PROBE, 0, False)
+ request = struct.pack(">B16sQI", MODE_PROBE, session_id, 0, len(encrypted)) + encrypted
+
+ try:
+ with socket.create_connection((host, port), timeout=timeout) as sock:
+ sock.settimeout(timeout)
+ sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
+ sock.sendall(request)
+ header = read_exact(sock, RESPONSE_HEADER_SIZE)
+ status, body_size = struct.unpack(">BI", header)
+ if body_size > MAX_RESPONSE_BODY:
+ raise ValueError(f"response body too large: {body_size}")
+ body = read_exact(sock, body_size) if body_size else b""
+ decoded = sha256_ctr_mask(body, session_id, MODE_PROBE, 0, True)
+ elapsed = int((time.monotonic() - started) * 1000)
+ if status == STATUS_OK and decoded == BP_PROBE:
+ return Result(port, "bp", elapsed, "valid BP probe echo")
+ return Result(
+ port,
+ "open",
+ elapsed,
+ f"non-BP response status={status} body={decoded[:16].hex()}",
+ )
+ except (ConnectionRefusedError, TimeoutError, socket.timeout):
+ return Result(port, "closed", int((time.monotonic() - started) * 1000))
+ except OSError as exc:
+ return Result(
+ port,
+ "closed",
+ int((time.monotonic() - started) * 1000),
+ str(exc),
+ )
+ except Exception as exc: # A TCP service answered, but not with a valid BP frame.
+ return Result(
+ port,
+ "open",
+ int((time.monotonic() - started) * 1000),
+ str(exc),
+ )
+
+
+def parse_args() -> argparse.Namespace:
+ parser = argparse.ArgumentParser(
+ description=(
+ "Test TCP ports for a valid BP probe response. Only scan hosts you own "
+ "or have explicit permission to test."
+ )
+ )
+ parser.add_argument("--host", required=True, help="authorized IPv4, IPv6, or hostname")
+ parser.add_argument("--start-port", type=int, default=1, help="first port (default: 1)")
+ parser.add_argument("--end-port", type=int, default=65535, help="last port (default: 65535)")
+ parser.add_argument("--threads", type=int, default=1, help="maximum concurrent probes, 1-64 (default: 1)")
+ parser.add_argument(
+ "--delay-ms",
+ type=int,
+ default=200,
+ help="global delay between probe starts, 0-30000 ms (default: 200)",
+ )
+ parser.add_argument("--timeout", type=float, default=2.5, help="per-port timeout in seconds (default: 2.5)")
+ parser.add_argument("--show-open", action="store_true", help="also print open ports that do not speak BP")
+ parser.add_argument("--progress-every", type=int, default=1000, help="progress interval; 0 disables")
+ parser.add_argument("--output", type=Path, help="optional CSV output for BP and other open ports")
+ args = parser.parse_args()
+
+ if not 1 <= args.start_port <= 65535:
+ parser.error("--start-port must be between 1 and 65535")
+ if not 1 <= args.end_port <= 65535:
+ parser.error("--end-port must be between 1 and 65535")
+ if args.start_port > args.end_port:
+ parser.error("--start-port must not exceed --end-port")
+ if not 1 <= args.threads <= 64:
+ parser.error("--threads must be between 1 and 64")
+ if not 0 <= args.delay_ms <= 30000:
+ parser.error("--delay-ms must be between 0 and 30000")
+ if not 0.05 <= args.timeout <= 120:
+ parser.error("--timeout must be between 0.05 and 120 seconds")
+ if args.progress_every < 0:
+ parser.error("--progress-every must be 0 or greater")
+ return args
+
+
+def main() -> int:
+ args = parse_args()
+ total = args.end_port - args.start_port + 1
+ delay = args.delay_ms / 1000.0
+ completed = 0
+ results: list[Result] = []
+ started = time.monotonic()
+ print(
+ f"BP scan host={args.host} ports={args.start_port}-{args.end_port} "
+ f"threads={args.threads} delay={args.delay_ms}ms timeout={args.timeout:g}s"
+ )
+
+ print_lock = threading.Lock()
+
+ def consume(result: Result) -> None:
+ nonlocal completed
+ completed += 1
+ if result.state != "closed":
+ results.append(result)
+ with print_lock:
+ if result.state == "bp":
+ print(f"BP {args.host}:{result.port} {result.elapsed_ms}ms")
+ elif result.state == "open" and args.show_open:
+ suffix = f" {result.detail}" if result.detail else ""
+ print(f"OPEN {args.host}:{result.port} {result.elapsed_ms}ms{suffix}")
+ if args.progress_every and completed % args.progress_every == 0:
+ elapsed = time.monotonic() - started
+ print(f"progress {completed}/{total} elapsed={elapsed:.1f}s")
+
+ pending: set[concurrent.futures.Future[Result]] = set()
+ try:
+ with concurrent.futures.ThreadPoolExecutor(max_workers=args.threads) as pool:
+ for port in range(args.start_port, args.end_port + 1):
+ while len(pending) >= args.threads:
+ done, pending = concurrent.futures.wait(
+ pending,
+ return_when=concurrent.futures.FIRST_COMPLETED,
+ )
+ for future in done:
+ consume(future.result())
+ pending.add(pool.submit(test_port, args.host, port, args.timeout))
+ if delay:
+ time.sleep(delay)
+ for future in concurrent.futures.as_completed(pending):
+ consume(future.result())
+ except KeyboardInterrupt:
+ print("\nInterrupted; partial results follow.", file=sys.stderr)
+
+ results.sort(key=lambda item: item.port)
+ bp_ports = [item.port for item in results if item.state == "bp"]
+ elapsed = time.monotonic() - started
+ print(f"completed={completed}/{total} elapsed={elapsed:.1f}s")
+ print("BP ports: " + (", ".join(map(str, bp_ports)) if bp_ports else "none"))
+
+ if args.output:
+ with args.output.open("w", newline="", encoding="utf-8") as handle:
+ writer = csv.writer(handle)
+ writer.writerow(("host", "port", "state", "elapsed_ms", "detail"))
+ for result in results:
+ writer.writerow((args.host, result.port, result.state, result.elapsed_ms, result.detail))
+ print(f"wrote {args.output}")
+
+ return 0 if bp_ports else 1
+
+
+if __name__ == "__main__":
+ raise SystemExit(main())
diff --git a/core/cmd/dragontcp-client/bp.go b/core/cmd/dragontcp-client/bp.go
new file mode 100644
index 0000000..cd38371
--- /dev/null
+++ b/core/cmd/dragontcp-client/bp.go
@@ -0,0 +1,499 @@
+package main
+
+import (
+ "encoding/binary"
+ "fmt"
+ "io"
+ "net"
+ "sync"
+ "time"
+
+ "dragontcp/internal/cover"
+ "dragontcp/internal/protocol"
+ "dragontcp/internal/wire"
+)
+
+const (
+ bpModeProbe byte = 0
+ bpModeUpload byte = 1
+ bpModeDownload byte = 2
+ bpModeBatchDownload byte = 3
+ bpModeACK byte = 4
+ bpHeaderSize = 29
+)
+
+var bpOpenMagic = [4]byte{'D', 'O', 'P', '1'}
+var bpCloseMagic = [4]byte{'D', 'C', 'L', '1'}
+
+type bpPhysicalConn struct {
+ conn net.Conn
+ requests int
+}
+
+type bpLane struct {
+ mu sync.Mutex
+ serverAddr string
+ tcpBuffer int
+ reconnectEvery int
+ timeout time.Duration
+ coverProfile cover.Profile
+ autoReconnect bool
+ pc *bpPhysicalConn
+ closed bool
+}
+
+func newBPLane(serverAddr string, tcpBuffer, reconnectEvery int, timeout time.Duration, coverProfile cover.Profile) *bpLane {
+ autoReconnect := reconnectEvery == 1
+ if autoReconnect {
+ reconnectEvery = 0
+ }
+ return &bpLane{
+ serverAddr: serverAddr,
+ tcpBuffer: tcpBuffer,
+ reconnectEvery: reconnectEvery,
+ timeout: timeout,
+ coverProfile: coverProfile,
+ autoReconnect: autoReconnect,
+ }
+}
+
+func (l *bpLane) transportFailureLocked(reused bool) {
+ if l.autoReconnect && reused {
+ l.reconnectEvery = 1
+ }
+ l.discardLocked()
+}
+
+func (l *bpLane) discardLocked() {
+ if l.pc != nil {
+ _ = l.pc.conn.Close()
+ l.pc = nil
+ }
+}
+
+func (l *bpLane) closeAfterLocked() {
+ if l.pc != nil && l.reconnectEvery > 0 && l.pc.requests >= l.reconnectEvery {
+ l.discardLocked()
+ }
+}
+
+func (l *bpLane) ensureLocked() error {
+ if l.closed {
+ return net.ErrClosed
+ }
+ if l.pc != nil {
+ if l.reconnectEvery <= 0 || l.pc.requests < l.reconnectEvery {
+ return nil
+ }
+ l.discardLocked()
+ }
+ d := net.Dialer{Timeout: 10 * time.Second, KeepAlive: 30 * time.Second}
+ conn, err := d.Dial("tcp", l.serverAddr)
+ if err != nil {
+ return err
+ }
+ if err := cover.WritePreface(conn, l.coverProfile); err != nil {
+ _ = conn.Close()
+ return err
+ }
+ protocol.TuneTCP(conn)
+ protocol.TuneTCPBuffer(conn, l.tcpBuffer)
+ l.pc = &bpPhysicalConn{conn: conn}
+ return nil
+}
+
+func (l *bpLane) Close() {
+ l.mu.Lock()
+ l.closed = true
+ l.discardLocked()
+ l.mu.Unlock()
+}
+
+func writeBPRequest(w io.Writer, mode byte, sid wire.SessionID, seq uint64, payload []byte, downloadHint uint32, headerMask byte, clear bool) error {
+ n := uint32(len(payload))
+ if mode == bpModeDownload {
+ n = downloadHint
+ payload = nil
+ }
+ if len(payload) > wire.MaxPayload {
+ return fmt.Errorf("BP payload too large: %d", len(payload))
+ }
+ var header [bpHeaderSize]byte
+ header[0] = mode ^ headerMask
+ copy(header[1:17], sid[:])
+ binary.BigEndian.PutUint64(header[17:25], seq)
+ binary.BigEndian.PutUint32(header[25:29], n)
+ if clear {
+ buffers := net.Buffers{header[:], payload}
+ _, err := buffers.WriteTo(w)
+ return err
+ }
+ packet := make([]byte, bpHeaderSize+len(payload))
+ copy(packet[:bpHeaderSize], header[:])
+ copy(packet[bpHeaderSize:], payload)
+ wire.MaskInPlace(packet[bpHeaderSize:], sid, mode, seq, false)
+ for len(packet) > 0 {
+ written, err := w.Write(packet)
+ if err != nil {
+ return err
+ }
+ if written <= 0 {
+ return io.ErrShortWrite
+ }
+ packet = packet[written:]
+ }
+ return nil
+}
+
+func readBPResponse(r io.Reader, sid wire.SessionID, mode byte, seq uint64, headerMask byte, clear bool) (byte, []byte, error) {
+ status, body, err := wire.ReadResponseProfile(r, headerMask)
+ if err == nil && status != wire.StatusError && len(body) > 0 && !clear {
+ wire.MaskInPlace(body, sid, mode, seq, true)
+ }
+ return status, body, err
+}
+
+func (l *bpLane) single(mode byte, sid wire.SessionID, seq uint64, payload []byte, downloadHint uint32) (byte, []byte, error) {
+ l.mu.Lock()
+ defer l.mu.Unlock()
+ timeout := l.timeout
+ if timeout <= 0 {
+ timeout = 5 * time.Second
+ }
+ var lastErr error
+ for attempt := 0; attempt < 2; attempt++ {
+ if err := l.ensureLocked(); err != nil {
+ lastErr = err
+ continue
+ }
+ reused := l.pc.requests > 0
+ _ = l.pc.conn.SetDeadline(time.Now().Add(timeout))
+ if err := writeBPRequest(l.pc.conn, mode, sid, seq, payload, downloadHint, l.coverProfile.HeaderMask, l.coverProfile.Clear); err != nil {
+ lastErr = err
+ l.transportFailureLocked(reused)
+ continue
+ }
+ status, body, err := readBPResponse(l.pc.conn, sid, mode, seq, l.coverProfile.HeaderMask, l.coverProfile.Clear)
+ if err != nil {
+ lastErr = err
+ l.transportFailureLocked(reused)
+ continue
+ }
+ l.pc.requests++
+ _ = l.pc.conn.SetDeadline(time.Time{})
+ l.closeAfterLocked()
+ return status, body, nil
+ }
+ return 0, nil, fmt.Errorf("BP request failed after reconnect: %w", lastErr)
+}
+
+func decodeBPData(body []byte) ([]byte, error) {
+ if len(body) < 4 {
+ return nil, fmt.Errorf("short BP DATA body")
+ }
+ n := int(binary.BigEndian.Uint32(body[:4]))
+ if n < 0 || n > len(body)-4 {
+ return nil, fmt.Errorf("bad BP DATA length")
+ }
+ return append([]byte(nil), body[4:4+n]...), nil
+}
+
+func (l *bpLane) download(sid wire.SessionID, offset uint64, maxChunk, count int) ([][]byte, byte, error) {
+ l.mu.Lock()
+ defer l.mu.Unlock()
+ timeout := l.timeout
+ if timeout <= 0 {
+ timeout = 5 * time.Second
+ }
+ mode := bpModeDownload
+ payload := []byte(nil)
+ hint := uint32(maxChunk)
+ if count > 1 {
+ mode = bpModeBatchDownload
+ payload = make([]byte, 6)
+ binary.BigEndian.PutUint32(payload[:4], uint32(maxChunk))
+ binary.BigEndian.PutUint16(payload[4:6], uint16(count))
+ hint = 0
+ }
+ responses := 1
+ if mode == bpModeBatchDownload {
+ responses = count
+ }
+ var lastErr error
+ for attempt := 0; attempt < 2; attempt++ {
+ if err := l.ensureLocked(); err != nil {
+ lastErr = err
+ continue
+ }
+ reused := l.pc.requests > 0
+ _ = l.pc.conn.SetDeadline(time.Now().Add(timeout))
+ if err := writeBPRequest(l.pc.conn, mode, sid, offset, payload, hint, l.coverProfile.HeaderMask, l.coverProfile.Clear); err != nil {
+ lastErr = err
+ l.transportFailureLocked(reused)
+ continue
+ }
+
+ out := make([][]byte, 0, responses)
+ lastStatus := wire.StatusOK
+ for i := 0; i < responses; i++ {
+ status, body, err := readBPResponse(l.pc.conn, sid, mode, offset, l.coverProfile.HeaderMask, l.coverProfile.Clear)
+ if err != nil {
+ lastErr = err
+ l.transportFailureLocked(reused)
+ goto retry
+ }
+ lastStatus = status
+ switch status {
+ case wire.StatusData:
+ data, err := decodeBPData(body)
+ if err != nil {
+ l.discardLocked()
+ return out, status, err
+ }
+ if len(data) > 0 {
+ out = append(out, data)
+ }
+ case wire.StatusOK, wire.StatusWait:
+ case wire.StatusEOF:
+ case wire.StatusError:
+ l.discardLocked()
+ return out, status, fmt.Errorf("%s", string(body))
+ default:
+ l.discardLocked()
+ return out, status, fmt.Errorf("unexpected BP download status %d", status)
+ }
+ }
+ l.pc.requests++
+ _ = l.pc.conn.SetDeadline(time.Time{})
+ l.closeAfterLocked()
+ return out, lastStatus, nil
+ retry:
+ }
+ return nil, 0, fmt.Errorf("BP download request failed after reconnect: %w", lastErr)
+}
+
+type bpConn struct {
+ sid wire.SessionID
+ opts chunkClientOptions
+ uploadLane *bpLane
+ downloadLane *bpLane
+ upSizer *adaptiveSizer
+ downSizer *adaptiveSizer
+
+ writeMu sync.Mutex
+ upOffset uint64
+
+ readMu sync.Mutex
+ readBuf []byte
+ downloadOffset uint64
+ consumedOffset uint64
+ lastAck uint64
+ eof bool
+ pipeline int
+
+ closeOnce sync.Once
+}
+
+func openBPTunnel(serverAddr, token, targetHost string, targetPort int, opts chunkClientOptions) (net.Conn, error) {
+ if opts.minSize < 32 {
+ opts.minSize = 32
+ }
+ if opts.maxSize < opts.minSize {
+ opts.maxSize = opts.minSize
+ }
+ if opts.maxSize > 1024*1024 {
+ opts.maxSize = 1024 * 1024
+ }
+ if opts.startSize < opts.minSize || opts.startSize > opts.maxSize {
+ opts.startSize = opts.maxSize
+ }
+ if opts.txnTimeout <= 0 {
+ opts.txnTimeout = 5 * time.Second
+ }
+ if opts.maxPipeline < 1 {
+ opts.maxPipeline = 1
+ }
+ if opts.maxPipeline > 256 {
+ opts.maxPipeline = 256
+ }
+ if opts.minPipeline < 1 {
+ opts.minPipeline = 1
+ }
+ if opts.minPipeline > opts.maxPipeline {
+ opts.minPipeline = opts.maxPipeline
+ }
+ reconnect := opts.reconnectEvery
+ if reconnect < 0 {
+ reconnect = 0
+ }
+
+ sid, err := randomSessionID()
+ if err != nil {
+ return nil, err
+ }
+ uploadLane := newBPLane(serverAddr, opts.tcpBuffer, reconnect, opts.txnTimeout, opts.coverProfile)
+ status, body, err := uploadLane.single(bpModeUpload, sid, 0, nil, 0)
+ if err != nil {
+ uploadLane.Close()
+ return nil, err
+ }
+ if status == wire.StatusError {
+ uploadLane.Close()
+ return nil, fmt.Errorf("%s", string(body))
+ }
+ if status != wire.StatusOK {
+ uploadLane.Close()
+ return nil, fmt.Errorf("bad BP registration response %d", status)
+ }
+ openPayload, err := encodeOpen(token, targetHost, targetPort)
+ if err != nil {
+ uploadLane.Close()
+ return nil, err
+ }
+ openPayload = append(append([]byte(nil), bpOpenMagic[:]...), openPayload...)
+ status, body, err = uploadLane.single(bpModeUpload, sid, 1, openPayload, 0)
+ if err != nil {
+ uploadLane.Close()
+ return nil, err
+ }
+ if status == wire.StatusError {
+ uploadLane.Close()
+ return nil, fmt.Errorf("%s", string(body))
+ }
+ if status != wire.StatusOK {
+ uploadLane.Close()
+ return nil, fmt.Errorf("bad BP OPEN response %d", status)
+ }
+
+ c := &bpConn{
+ sid: sid,
+ opts: opts,
+ uploadLane: uploadLane,
+ downloadLane: newBPLane(serverAddr, opts.tcpBuffer, reconnect, opts.txnTimeout, opts.coverProfile),
+ pipeline: opts.maxPipeline,
+ }
+ c.upSizer = newAdaptiveSizer("BP upload", opts.startSize, opts)
+ c.downSizer = newAdaptiveSizer("BP download", opts.startSize, opts)
+ return c, nil
+}
+
+func (c *bpConn) fillReadBuffer() error {
+ if c.eof {
+ return io.EOF
+ }
+ for len(c.readBuf) == 0 && !c.eof {
+ if c.consumedOffset > c.lastAck {
+ status, body, err := c.downloadLane.single(bpModeACK, c.sid, c.consumedOffset, nil, 0)
+ if err != nil {
+ return err
+ }
+ if status == wire.StatusError {
+ return fmt.Errorf("%s", string(body))
+ }
+ c.lastAck = c.consumedOffset
+ }
+ chunk := c.downSizer.Current()
+ count := c.pipeline
+ if count < c.opts.minPipeline {
+ count = c.opts.minPipeline
+ }
+ if count > c.opts.maxPipeline {
+ count = c.opts.maxPipeline
+ }
+ data, status, err := c.downloadLane.download(c.sid, c.downloadOffset, chunk, count)
+ if err != nil {
+ old, next := c.downSizer.FailureReason(chunk, err)
+ if old == next && next == c.opts.minSize {
+ return err
+ }
+ time.Sleep(30 * time.Millisecond)
+ continue
+ }
+ c.readBuf = appendChunkParts(c.readBuf, data)
+ for _, part := range data {
+ c.downloadOffset += uint64(len(part))
+ }
+ if len(data) > 0 {
+ c.downSizer.Success(chunk)
+ if c.pipeline < c.opts.maxPipeline {
+ c.pipeline++
+ }
+ }
+ if status == wire.StatusEOF {
+ c.eof = true
+ }
+ if len(c.readBuf) == 0 && !c.eof {
+ delay := c.opts.pollDelay
+ if delay <= 0 {
+ delay = 5 * time.Millisecond
+ }
+ time.Sleep(delay)
+ }
+ }
+ if c.eof && len(c.readBuf) == 0 {
+ return io.EOF
+ }
+ return nil
+}
+
+func (c *bpConn) Read(p []byte) (int, error) {
+ c.readMu.Lock()
+ defer c.readMu.Unlock()
+ if len(p) == 0 {
+ return 0, nil
+ }
+ if len(c.readBuf) == 0 {
+ if err := c.fillReadBuffer(); err != nil {
+ return 0, err
+ }
+ }
+ n := copy(p, c.readBuf)
+ c.readBuf = c.readBuf[n:]
+ c.consumedOffset += uint64(n)
+ return n, nil
+}
+
+func (c *bpConn) Write(p []byte) (int, error) {
+ c.writeMu.Lock()
+ defer c.writeMu.Unlock()
+ total := 0
+ for len(p) > 0 {
+ size := c.upSizer.Current()
+ n := minInt(size, len(p))
+ status, body, err := c.uploadLane.single(bpModeUpload, c.sid, c.upOffset+2, p[:n], 0)
+ if err != nil {
+ old, next := c.upSizer.FailureReason(size, err)
+ if old == next && next == c.opts.minSize {
+ return total, err
+ }
+ time.Sleep(30 * time.Millisecond)
+ continue
+ }
+ if status == wire.StatusError {
+ return total, fmt.Errorf("%s", string(body))
+ }
+ if status != wire.StatusOK {
+ return total, fmt.Errorf("unexpected BP upload status %d", status)
+ }
+ c.upOffset += uint64(n)
+ total += n
+ p = p[n:]
+ c.upSizer.Success(size)
+ }
+ return total, nil
+}
+
+func (c *bpConn) Close() error {
+ c.closeOnce.Do(func() {
+ _, _, _ = c.downloadLane.single(bpModeACK, c.sid, c.consumedOffset, bpCloseMagic[:], 0)
+ c.uploadLane.Close()
+ c.downloadLane.Close()
+ })
+ return nil
+}
+
+func (c *bpConn) LocalAddr() net.Addr { return dummyAddr("dragontcp-bp-local") }
+func (c *bpConn) RemoteAddr() net.Addr { return dummyAddr("dragontcp-bp-remote") }
+func (c *bpConn) SetDeadline(time.Time) error { return nil }
+func (c *bpConn) SetReadDeadline(time.Time) error { return nil }
+func (c *bpConn) SetWriteDeadline(time.Time) error { return nil }
diff --git a/core/cmd/dragontcp-client/chunk.go b/core/cmd/dragontcp-client/chunk.go
index dfee850..67f1da2 100644
--- a/core/cmd/dragontcp-client/chunk.go
+++ b/core/cmd/dragontcp-client/chunk.go
@@ -11,6 +11,7 @@ import (
"sync/atomic"
"time"
+ "dragontcp/internal/cover"
"dragontcp/internal/protocol"
"dragontcp/internal/wire"
)
@@ -29,6 +30,9 @@ type chunkClientOptions struct {
tcpBuffer int
minPipeline int
maxPipeline int
+ headerMask byte
+ coverProfile cover.Profile
+ skipPathProbe bool
}
type adaptiveSizer struct {
@@ -120,6 +124,10 @@ func (s *adaptiveSizer) Success(attempted int) {
}
func (s *adaptiveSizer) Failure(attempted int) (int, int) {
+ return s.FailureReason(attempted, nil)
+}
+
+func (s *adaptiveSizer) FailureReason(attempted int, cause error) (int, int) {
s.mu.Lock()
defer s.mu.Unlock()
old := s.current
@@ -147,7 +155,11 @@ func (s *adaptiveSizer) Failure(attempted int) (int, int) {
}
s.current = next
if s.logChanges && old != next {
- fmt.Printf("adaptive %s chunk: %d -> %d after transport failure\n", s.name, old, next)
+ if cause != nil {
+ fmt.Printf("adaptive %s chunk: %d -> %d after transport failure: %v\n", s.name, old, next, cause)
+ } else {
+ fmt.Printf("adaptive %s chunk: %d -> %d after transport failure\n", s.name, old, next)
+ }
}
return old, next
}
@@ -163,19 +175,39 @@ type requestLane struct {
tcpBuffer int
reconnectEvery int
timeout time.Duration
+ headerMask byte
+ coverProfile cover.Profile
+ autoReconnect bool
pc *physicalConn
closed bool
}
-func newRequestLane(serverAddr string, tcpBuffer, reconnectEvery int, timeout time.Duration) *requestLane {
+func newRequestLane(serverAddr string, tcpBuffer, reconnectEvery int, timeout time.Duration, headerMask byte, coverProfile cover.Profile) *requestLane {
+ autoReconnect := reconnectEvery == 1
+ if autoReconnect {
+ // Auto starts persistent. If a request fails only after this lane has
+ // already completed traffic on the connection, it learns that reuse is
+ // unsafe and switches itself to one request per connection.
+ reconnectEvery = 0
+ }
return &requestLane{
serverAddr: serverAddr,
tcpBuffer: tcpBuffer,
reconnectEvery: reconnectEvery,
timeout: timeout,
+ headerMask: headerMask,
+ coverProfile: coverProfile,
+ autoReconnect: autoReconnect,
}
}
+func (l *requestLane) transportFailureLocked(reused bool) {
+ if l.autoReconnect && reused {
+ l.reconnectEvery = 1
+ }
+ l.discardLocked()
+}
+
func (l *requestLane) discardLocked() {
if l.pc != nil {
_ = l.pc.conn.Close()
@@ -204,6 +236,10 @@ func (l *requestLane) ensureLocked() error {
if err != nil {
return err
}
+ if err := cover.WritePreface(conn, l.coverProfile); err != nil {
+ _ = conn.Close()
+ return err
+ }
protocol.TuneTCP(conn)
protocol.TuneTCPBuffer(conn, l.tcpBuffer)
l.pc = &physicalConn{conn: conn}
@@ -220,30 +256,38 @@ func (l *requestLane) Close() {
func (l *requestLane) single(mode byte, sid wire.SessionID, seq uint64, payload []byte) (byte, []byte, error) {
l.mu.Lock()
defer l.mu.Unlock()
- if err := l.ensureLocked(); err != nil {
- return 0, nil, err
- }
timeout := l.timeout
if timeout <= 0 {
timeout = 5 * time.Second
}
- _ = l.pc.conn.SetDeadline(time.Now().Add(timeout))
- if err := wire.WriteRequest(l.pc.conn, mode, sid, seq, payload); err != nil {
- l.discardLocked()
- return 0, nil, err
+ var lastErr error
+ for attempt := 0; attempt < 2; attempt++ {
+ if err := l.ensureLocked(); err != nil {
+ lastErr = err
+ continue
+ }
+ reused := l.pc.requests > 0
+ _ = l.pc.conn.SetDeadline(time.Now().Add(timeout))
+ if err := wire.WriteRequestProfileEncoding(l.pc.conn, mode, sid, seq, payload, l.headerMask, l.coverProfile.Clear); err != nil {
+ lastErr = err
+ l.transportFailureLocked(reused)
+ continue
+ }
+ status, body, err := wire.ReadResponseProfile(l.pc.conn, l.headerMask)
+ if err != nil {
+ lastErr = err
+ l.transportFailureLocked(reused)
+ continue
+ }
+ l.pc.requests++
+ _ = l.pc.conn.SetDeadline(time.Time{})
+ l.closeAfterLocked()
+ if status != wire.StatusError && len(body) > 0 && !l.coverProfile.Clear {
+ body = wire.DecodeMaskedResponse(status, body, sid, mode, seq)
+ }
+ return status, body, nil
}
- status, body, err := wire.ReadResponse(l.pc.conn)
- if err != nil {
- l.discardLocked()
- return 0, nil, err
- }
- l.pc.requests++
- _ = l.pc.conn.SetDeadline(time.Time{})
- l.closeAfterLocked()
- if status != wire.StatusError && len(body) > 0 {
- body = wire.DecodeMaskedResponse(status, body, sid, mode, seq)
- }
- return status, body, nil
+ return 0, nil, fmt.Errorf("request failed after reconnect: %w", lastErr)
}
// download sends one compact request and consumes up to count response records.
@@ -252,61 +296,71 @@ func (l *requestLane) single(mode byte, sid wire.SessionID, seq uint64, payload
func (l *requestLane) download(sid wire.SessionID, startOffset, ackOffset uint64, maxChunk, count int) ([][]byte, byte, error) {
l.mu.Lock()
defer l.mu.Unlock()
- if err := l.ensureLocked(); err != nil {
- return nil, 0, err
- }
timeout := l.timeout
if timeout <= 0 {
timeout = 5 * time.Second
}
- _ = l.pc.conn.SetDeadline(time.Now().Add(timeout))
-
payload := make([]byte, 14)
binary.BigEndian.PutUint64(payload[0:8], ackOffset)
binary.BigEndian.PutUint32(payload[8:12], uint32(maxChunk))
binary.BigEndian.PutUint16(payload[12:14], uint16(count))
- if err := wire.WriteRequest(l.pc.conn, wire.ModeDownload, sid, startOffset, payload); err != nil {
- l.discardLocked()
- return nil, 0, err
- }
-
- out := make([][]byte, 0, count)
- offset := startOffset
- lastStatus := wire.StatusOK
- for i := 0; i < count; i++ {
- status, body, err := wire.ReadResponse(l.pc.conn)
- if err != nil {
- l.discardLocked()
- return out, lastStatus, err
+ var lastErr error
+ for attempt := 0; attempt < 2; attempt++ {
+ if err := l.ensureLocked(); err != nil {
+ lastErr = err
+ continue
}
- lastStatus = status
- switch status {
- case wire.StatusData:
- body = wire.DecodeMaskedResponse(status, body, sid, wire.ModeDownload, offset)
- if len(body) == 0 {
+ reused := l.pc.requests > 0
+ _ = l.pc.conn.SetDeadline(time.Now().Add(timeout))
+ if err := wire.WriteRequestProfileEncoding(l.pc.conn, wire.ModeDownload, sid, startOffset, payload, l.headerMask, l.coverProfile.Clear); err != nil {
+ lastErr = err
+ l.transportFailureLocked(reused)
+ continue
+ }
+
+ out := make([][]byte, 0, count)
+ offset := startOffset
+ lastStatus := wire.StatusOK
+ for i := 0; i < count; i++ {
+ status, body, err := wire.ReadResponseProfile(l.pc.conn, l.headerMask)
+ if err != nil {
+ lastErr = err
+ l.transportFailureLocked(reused)
+ goto retry
+ }
+ lastStatus = status
+ switch status {
+ case wire.StatusData:
+ if !l.coverProfile.Clear {
+ body = wire.DecodeMaskedResponse(status, body, sid, wire.ModeDownload, offset)
+ }
+ if len(body) == 0 {
+ l.discardLocked()
+ return out, status, fmt.Errorf("empty DATA response")
+ }
+ out = append(out, body)
+ offset += uint64(len(body))
+ case wire.StatusWait, wire.StatusEOF:
+ i = count // stop after this response
+ case wire.StatusError:
l.discardLocked()
- return out, status, fmt.Errorf("empty DATA response")
+ return out, status, fmt.Errorf("%s", string(body))
+ default:
+ l.discardLocked()
+ return out, status, fmt.Errorf("unknown response status %d", status)
+ }
+ if status == wire.StatusWait || status == wire.StatusEOF {
+ break
}
- out = append(out, body)
- offset += uint64(len(body))
- case wire.StatusWait, wire.StatusEOF:
- i = count // stop after this response
- case wire.StatusError:
- l.discardLocked()
- return out, status, fmt.Errorf("%s", string(body))
- default:
- l.discardLocked()
- return out, status, fmt.Errorf("unknown response status %d", status)
}
- if status == wire.StatusWait || status == wire.StatusEOF {
- break
- }
- }
- l.pc.requests++
- _ = l.pc.conn.SetDeadline(time.Time{})
- l.closeAfterLocked()
- return out, lastStatus, nil
+ l.pc.requests++
+ _ = l.pc.conn.SetDeadline(time.Time{})
+ l.closeAfterLocked()
+ return out, lastStatus, nil
+ retry:
+ }
+ return nil, 0, fmt.Errorf("download request failed after reconnect: %w", lastErr)
}
type pathProfile struct {
@@ -364,7 +418,7 @@ func probeOne(serverAddr, token string, opts chunkClientOptions, kind byte, cand
if timeout <= 0 || timeout > 2500*time.Millisecond {
timeout = 2500 * time.Millisecond
}
- lane := newRequestLane(serverAddr, opts.tcpBuffer, 1, timeout)
+ lane := newRequestLane(serverAddr, opts.tcpBuffer, 1, timeout, opts.headerMask, opts.coverProfile)
defer lane.Close()
seq := probeSeq.Add(1)
@@ -405,7 +459,7 @@ func probePersistent(serverAddr, token string, opts chunkClientOptions) bool {
if timeout <= 0 || timeout > 2500*time.Millisecond {
timeout = 2500 * time.Millisecond
}
- lane := newRequestLane(serverAddr, opts.tcpBuffer, 0, timeout)
+ lane := newRequestLane(serverAddr, opts.tcpBuffer, 0, timeout, opts.headerMask, opts.coverProfile)
defer lane.Close()
for i := 0; i < 8; i++ {
seq := probeSeq.Add(1)
@@ -459,7 +513,7 @@ func probeMaximum(serverAddr, token string, opts chunkClientOptions, kind byte)
}
func getPathProfile(serverAddr, token string, opts chunkClientOptions) pathProfile {
- key := fmt.Sprintf("%s|%s|%d|%d", serverAddr, token, opts.minSize, opts.maxSize)
+ key := fmt.Sprintf("%s|%s|%d|%d|%02x|%t|%04x|%d|%t", serverAddr, token, opts.minSize, opts.maxSize, opts.headerMask, opts.coverProfile.Enabled, opts.coverProfile.ID, opts.coverProfile.Padding, opts.coverProfile.Clear)
profileState.Lock()
if profileState.key == key && time.Since(profileState.p.at) < 30*time.Minute {
p := profileState.p
@@ -533,6 +587,31 @@ type chunkConn struct {
closeOnce sync.Once
}
+// appendChunkParts keeps the single-response fast path zero-copy. For a batch,
+// it reserves the complete size once rather than repeatedly growing and copying
+// the aggregate read buffer.
+func appendChunkParts(dst []byte, parts [][]byte) []byte {
+ if len(parts) == 0 {
+ return dst
+ }
+ if len(dst) == 0 && len(parts) == 1 {
+ return parts[0]
+ }
+ total := len(dst)
+ for _, part := range parts {
+ total += len(part)
+ }
+ if cap(dst) < total {
+ grown := make([]byte, len(dst), total)
+ copy(grown, dst)
+ dst = grown
+ }
+ for _, part := range parts {
+ dst = append(dst, part...)
+ }
+ return dst
+}
+
func openChunkTunnel(serverAddr, token, targetHost string, targetPort int, opts chunkClientOptions) (net.Conn, error) {
if opts.minSize < 32 {
opts.minSize = 32
@@ -565,16 +644,21 @@ func openChunkTunnel(serverAddr, token, targetHost string, targetPort int, opts
opts.minPipeline = opts.maxPipeline
}
- profile := getPathProfile(serverAddr, token, opts)
+ profile := pathProfile{
+ upload: opts.minSize,
+ download: opts.minSize,
+ persistent: false,
+ at: time.Now(),
+ }
+ if !opts.skipPathProbe {
+ profile = getPathProfile(serverAddr, token, opts)
+ }
reconnect := opts.reconnectEvery
// Compatibility-friendly reconnect modes:
// 0 = persistent (CLI explicit)
- // 1 = auto: persistent when the path probe succeeds, otherwise one request/connection
+ // 1 = auto: start persistent, then learn one request/connection only if
+ // reuse fails during real traffic
// N>=2 = force connection rotation after N logical requests
- // Resolved silently: this runs once per proxied flow, so it must never log.
- if reconnect == 1 && profile.persistent {
- reconnect = 0
- }
sid, err := randomSessionID()
if err != nil {
@@ -583,7 +667,7 @@ func openChunkTunnel(serverAddr, token, targetHost string, targetPort int, opts
// OPEN rides the upload lane instead of a throwaway connection. A dedicated
// control connection cost one extra dial per proxied flow, which shows up on
// the server as connection churn on top of the steady-state count.
- uploadLane := newRequestLane(serverAddr, opts.tcpBuffer, reconnect, opts.txnTimeout)
+ uploadLane := newRequestLane(serverAddr, opts.tcpBuffer, reconnect, opts.txnTimeout, opts.headerMask, opts.coverProfile)
payload, err := encodeOpen(token, targetHost, targetPort)
if err != nil {
uploadLane.Close()
@@ -624,7 +708,7 @@ func openChunkTunnel(serverAddr, token, targetHost string, targetPort int, opts
opts: opts,
serverMax: serverMax,
uploadLane: uploadLane,
- downloadLane: newRequestLane(serverAddr, opts.tcpBuffer, reconnect, opts.txnTimeout),
+ downloadLane: newRequestLane(serverAddr, opts.tcpBuffer, reconnect, opts.txnTimeout, opts.headerMask, opts.coverProfile),
// Start at the configured ceiling. On transport failure the batch is
// halved but never below minPipeline; successful data grows it back by
// one. When min == max the depth is pinned and never adapts, which is
@@ -659,8 +743,8 @@ func (c *chunkConn) fillReadBuffer() error {
}
data, status, err := c.downloadLane.download(c.sid, c.downloadOffset, c.consumedOffset, chunk, count)
+ c.readBuf = appendChunkParts(c.readBuf, data)
for _, part := range data {
- c.readBuf = append(c.readBuf, part...)
c.downloadOffset += uint64(len(part))
}
if len(data) > 0 {
@@ -678,10 +762,10 @@ func (c *chunkConn) fillReadBuffer() error {
c.pipeline = c.minPipeline
}
if c.opts.adaptLog && old != c.pipeline {
- fmt.Printf("adaptive download pipeline: %d -> %d after transport failure\n", old, c.pipeline)
+ fmt.Printf("adaptive download pipeline: %d -> %d after transport failure: %v\n", old, c.pipeline, err)
}
} else {
- old, next := c.downSizer.Failure(chunk)
+ old, next := c.downSizer.FailureReason(chunk, err)
if old == next && next == c.opts.minSize {
minFailures++
if minFailures >= 8 {
@@ -745,7 +829,7 @@ func (c *chunkConn) Write(p []byte) (int, error) {
n := minInt(size, len(p))
status, body, err := c.uploadLane.single(wire.ModeUpload, c.sid, c.upOffset, p[:n])
if err != nil {
- old, next := c.upSizer.Failure(size)
+ old, next := c.upSizer.FailureReason(size, err)
if old == next && next == c.opts.minSize {
minFailures++
if minFailures >= 8 {
diff --git a/core/cmd/dragontcp-client/chunk_test.go b/core/cmd/dragontcp-client/chunk_test.go
index 16f0a24..9f76d97 100644
--- a/core/cmd/dragontcp-client/chunk_test.go
+++ b/core/cmd/dragontcp-client/chunk_test.go
@@ -1,6 +1,13 @@
package main
-import "testing"
+import (
+ "net"
+ "testing"
+ "time"
+
+ "dragontcp/internal/cover"
+ "dragontcp/internal/wire"
+)
func TestAdaptiveSizerRecoversFromMinimum(t *testing.T) {
opts := chunkClientOptions{
@@ -23,9 +30,72 @@ func TestAdaptiveSizerRecoversFromMinimum(t *testing.T) {
}
}
+func TestReconnectAutoLearnsFromRealReuseFailure(t *testing.T) {
+ ln, err := net.Listen("tcp", "127.0.0.1:0")
+ if err != nil {
+ t.Fatal(err)
+ }
+ defer ln.Close()
+ serverErr := make(chan error, 1)
+ go func() {
+ first, err := ln.Accept()
+ if err != nil {
+ serverErr <- err
+ return
+ }
+ if _, err := wire.ReadRequest(first); err != nil {
+ serverErr <- err
+ return
+ }
+ if err := wire.WriteResponse(first, wire.StatusOK, nil); err != nil {
+ serverErr <- err
+ return
+ }
+ _ = first.Close() // Force the next logical request to reconnect.
+
+ second, err := ln.Accept()
+ if err != nil {
+ serverErr <- err
+ return
+ }
+ defer second.Close()
+ if _, err := wire.ReadRequest(second); err != nil {
+ serverErr <- err
+ return
+ }
+ serverErr <- wire.WriteResponse(second, wire.StatusOK, nil)
+ }()
+
+ lane := newRequestLane(ln.Addr().String(), 0, 1, time.Second, 0, cover.Profile{})
+ defer lane.Close()
+ if !lane.autoReconnect || lane.reconnectEvery != 0 {
+ t.Fatalf("auto lane started auto=%t reconnectEvery=%d", lane.autoReconnect, lane.reconnectEvery)
+ }
+ var sid wire.SessionID
+ if status, _, err := lane.single(wire.ModeProbe, sid, 1, nil); err != nil || status != wire.StatusOK {
+ t.Fatalf("first request status=%d err=%v", status, err)
+ }
+ if status, _, err := lane.single(wire.ModeProbe, sid, 2, nil); err != nil || status != wire.StatusOK {
+ t.Fatalf("retried request status=%d err=%v", status, err)
+ }
+ if lane.reconnectEvery != 1 || lane.pc != nil {
+ t.Fatalf("auto lane did not learn single-request mode: reconnectEvery=%d pc=%v", lane.reconnectEvery, lane.pc)
+ }
+ if err := <-serverErr; err != nil {
+ t.Fatal(err)
+ }
+}
+
func TestReconnectZeroMeansPersistent(t *testing.T) {
- lane := newRequestLane("127.0.0.1:1", 0, 0, 0)
+ lane := newRequestLane("127.0.0.1:1", 0, 0, 0, 0, cover.Profile{})
if lane.reconnectEvery != 0 {
t.Fatalf("reconnectEvery=%d, want 0", lane.reconnectEvery)
}
}
+
+func TestBPAutoStartsPersistent(t *testing.T) {
+ lane := newBPLane("127.0.0.1:1", 0, 1, time.Second, cover.Profile{})
+ if !lane.autoReconnect || lane.reconnectEvery != 0 {
+ t.Fatalf("BP auto lane started auto=%t reconnectEvery=%d", lane.autoReconnect, lane.reconnectEvery)
+ }
+}
diff --git a/core/cmd/dragontcp-client/main.go b/core/cmd/dragontcp-client/main.go
index 3b6215f..05e5420 100644
--- a/core/cmd/dragontcp-client/main.go
+++ b/core/cmd/dragontcp-client/main.go
@@ -377,10 +377,12 @@ func main() {
chunkPollers = flag.Int("chunk-pollers", 1, "reserved compatibility setting; binary transport uses one download worker")
chunkConcurrency = flag.Int("chunk-concurrency", 1, "maximum download records per request (1-256)")
chunkConcurrencyMin = flag.Int("chunk-concurrency-min", 1, "minimum download records per request (1-256); equal to --chunk-concurrency pins the depth")
- chunkReconnect = flag.Int("chunk-reconnect-every", 0, "force reconnect after N logical requests; 0 = persistent/automatic")
+ chunkReconnect = flag.Int("chunk-reconnect-every", 0, "connection reuse: 0 persistent, 1 auto-learn, N rotate after N requests")
chunkPollDelay = flag.Duration("chunk-poll-delay", 2*time.Millisecond, "delay after an empty chunk poll")
- chunkTimeout = flag.Duration("chunk-timeout", 2*time.Second, "per-record transaction timeout before adaptive shrink")
- wireMode = flag.String("wire", "auto", "wire mode: b, x, or auto (probe and pick)")
+ chunkTimeout = flag.Duration("chunk-timeout", 5*time.Second, "per-record transaction timeout before adaptive shrink")
+ wireMode = flag.String("wire", "auto", "wire mode: b, bp, x, or auto (probe and pick)")
+ wireProbeDelay = flag.Duration("wire-probe-delay", time.Second, "minimum delay between wire profile probe starts (200ms-30s)")
+ wireProbeThreads = flag.Int("wire-probe-threads", 1, "maximum concurrent wire profile probes (1-16)")
)
flag.Parse()
@@ -430,19 +432,29 @@ func main() {
}
*wireMode = strings.ToLower(strings.TrimSpace(*wireMode))
switch *wireMode {
- case WireBinary, WireXOR, WireAuto:
+ case WireBinary, WireBP, WireXOR, WireAuto:
case "binary":
*wireMode = WireBinary
+ case "bh", "h":
+ *wireMode = WireBP
case "xor":
*wireMode = WireXOR
default:
- fmt.Fprintln(os.Stderr, "--wire must be b, x or auto")
+ fmt.Fprintln(os.Stderr, "--wire must be b, bp, x or auto")
os.Exit(2)
}
if *chunkReconnect < 0 {
fmt.Fprintln(os.Stderr, "--chunk-reconnect-every must be 0 or greater")
os.Exit(2)
}
+ if *wireProbeDelay < 200*time.Millisecond || *wireProbeDelay > 30*time.Second {
+ fmt.Fprintln(os.Stderr, "--wire-probe-delay must be between 200ms and 30s")
+ os.Exit(2)
+ }
+ if *wireProbeThreads < 1 || *wireProbeThreads > 16 {
+ fmt.Fprintln(os.Stderr, "--wire-probe-threads must be between 1 and 16")
+ os.Exit(2)
+ }
chunkOpts := chunkClientOptions{
startSize: *chunkStart,
minSize: *chunkMin,
@@ -498,15 +510,11 @@ func main() {
)
}
- wires := newWireSelector(*wireMode, serverAddr, *token, chunkOpts, xorOpts)
- if *wireMode == WireAuto {
- fmt.Printf("wire=auto probing %s\n", probeHost)
- // Resolve in the background so startup is not blocked; a connection that
- // arrives first simply waits for the same result.
- go wires.mode()
- } else {
- fmt.Printf("wire=%s (manual)\n", *wireMode)
- }
+ wires := newWireSelector(*wireMode, serverAddr, *token, chunkOpts, xorOpts, *wireProbeDelay, *wireProbeThreads)
+ fmt.Printf("wire=%s discovering fixed header profile via http://%s/ probe_delay=%s probe_threads=%d\n", *wireMode, probeHost, wireProbeDelay.String(), *wireProbeThreads)
+ // Discover in the background so the local listener starts immediately. A
+ // connection arriving first waits on the same selector lock and result.
+ go wires.mode()
slots := make(chan struct{}, *maxConnections)
diff --git a/core/cmd/dragontcp-client/wireselect.go b/core/cmd/dragontcp-client/wireselect.go
index d45ca5f..0bcf6fe 100644
--- a/core/cmd/dragontcp-client/wireselect.go
+++ b/core/cmd/dragontcp-client/wireselect.go
@@ -1,18 +1,21 @@
package main
import (
+ "bufio"
"fmt"
"net"
"strings"
"sync"
"time"
+ "dragontcp/internal/cover"
"dragontcp/internal/xorchunk"
)
-// DragonTCP speaks two wires that are not interchangeable:
+// DragonTCP speaks three wires that are not interchangeable:
//
-// b — compact binary records (29/5-byte headers, SHA-256 keystream mask)
+// b — compact binary records (29/5-byte headers, clear or SHA-256-compatible payloads)
+// bp — compatible registration/upload/download/ACK records, clear or SHA-256-compatible
// x — legacy UP/OK framing with XOR 0xAD over ASCII chunk commands
//
// Networks differ in which they pass, so the client can be pinned to either or
@@ -20,6 +23,7 @@ import (
// keeping the first that answers.
const (
WireBinary = "b"
+ WireBP = "bp"
WireXOR = "x"
WireAuto = "auto"
)
@@ -34,109 +38,316 @@ const (
)
type wireSelector struct {
- mu sync.Mutex
- configured string // b, x or auto
- resolved string // b or x once decided
- serverAddr string
- token string
- binOpts chunkClientOptions
- xorOpts xorchunk.Options
+ mu sync.Mutex
+ configured string // b, x or auto
+ resolved wireChoice
+ hasChoice bool
+ serverAddr string
+ token string
+ binOpts chunkClientOptions
+ xorOpts xorchunk.Options
+ probeDelay time.Duration
+ probeThreads int
+
+ // Test hooks are nil in production.
+ candidateOverride []wireChoice
+ probeOverride func(wireChoice) bool
}
-func newWireSelector(configured, serverAddr, token string, binOpts chunkClientOptions, xorOpts xorchunk.Options) *wireSelector {
- s := &wireSelector{
- configured: configured,
- serverAddr: serverAddr,
- token: token,
- binOpts: binOpts,
- xorOpts: xorOpts,
+type wireChoice struct {
+ mode string
+ mask byte
+ cover cover.Profile
+}
+
+func (c wireChoice) String() string {
+ if c.mode == WireBP {
+ if c.cover.Enabled {
+ return fmt.Sprintf("bp/%s", c.cover)
+ }
+ return "bp/direct"
}
- if configured != WireAuto {
- s.resolved = configured
+ if c.cover.Enabled {
+ return fmt.Sprintf("%s/mask-%02x/%s", c.mode, c.mask, c.cover)
+ }
+ return fmt.Sprintf("%s/mask-%02x/direct", c.mode, c.mask)
+}
+
+func newWireSelector(configured, serverAddr, token string, binOpts chunkClientOptions, xorOpts xorchunk.Options, probeDelay time.Duration, probeThreads int) *wireSelector {
+ s := &wireSelector{
+ configured: configured,
+ serverAddr: serverAddr,
+ token: token,
+ binOpts: binOpts,
+ xorOpts: xorOpts,
+ probeDelay: probeDelay,
+ probeThreads: probeThreads,
}
return s
}
// dial opens a tunnel over the active wire, resolving the wire first if needed.
func (s *wireSelector) dial(host string, port int) (net.Conn, error) {
- mode := s.mode()
- if mode == WireXOR {
- return xorchunk.Open(s.serverAddr, s.token, host, port, s.xorOpts)
+ choice := s.mode()
+ if choice.mode == WireBP {
+ opts := s.binOpts
+ opts.headerMask = choice.mask
+ opts.coverProfile = choice.cover
+ return openBPTunnel(s.serverAddr, s.token, host, port, opts)
}
- return openChunkTunnel(s.serverAddr, s.token, host, port, s.binOpts)
+ if choice.mode == WireXOR {
+ if choice.cover.Enabled {
+ return xorchunk.Open(s.serverAddr, s.token, host, port, s.xorOpts.WithCoverProfile(choice.cover))
+ }
+ return xorchunk.Open(s.serverAddr, s.token, host, port, s.xorOpts.WithHeaderMask(choice.mask))
+ }
+ opts := s.binOpts
+ opts.headerMask = choice.mask
+ opts.coverProfile = choice.cover
+ return openChunkTunnel(s.serverAddr, s.token, host, port, opts)
}
// mode returns the wire to use, running detection once if configured as auto.
// Detection failure is not cached, so a client that starts before the network
// is usable retries on the next connection instead of latching a bad guess.
-func (s *wireSelector) mode() string {
+func (s *wireSelector) mode() wireChoice {
s.mu.Lock()
defer s.mu.Unlock()
- if s.resolved != "" {
+ if s.hasChoice {
return s.resolved
}
if picked, ok := s.detectLocked(); ok {
s.resolved = picked
+ s.hasChoice = true
return picked
}
- // Undecided: use the binary wire for this attempt without caching it.
- return WireBinary
+ // Undecided: honor an explicitly pinned family for this attempt without
+ // caching it. Auto retains the original B fallback and retries discovery on
+ // the next connection.
+ switch s.configured {
+ case WireBP:
+ return wireChoice{mode: WireBP}
+ case WireXOR:
+ return wireChoice{mode: WireXOR}
+ default:
+ return wireChoice{mode: WireBinary}
+ }
}
-func (s *wireSelector) detectLocked() (string, bool) {
- for _, candidate := range []string{WireBinary, WireXOR} {
- if s.probe(candidate) {
- fmt.Printf("wire probe: %s selected via %s\n", candidate, probeHost)
- return candidate, true
+// profileCandidates covers all compatible B first-byte bases and all X magic
+// masks that cannot be confused with B. Profile zero for each wire is first so
+// existing permissive networks complete discovery quickly.
+func (s *wireSelector) profileCandidates() []wireChoice {
+ var binaryProfiles []wireChoice
+ var xorProfiles []wireChoice
+ if s.configured == WireAuto || s.configured == WireBinary {
+ for n := 0; n < 256; n += 8 {
+ binaryProfiles = append(binaryProfiles, wireChoice{mode: WireBinary, mask: byte(n)})
}
- fmt.Printf("wire probe: %s failed\n", candidate)
}
- fmt.Printf("wire probe: neither wire reached %s; retrying later\n", probeHost)
- return "", false
+ if s.configured == WireAuto || s.configured == WireXOR {
+ for n := 0; n < 256; n++ {
+ mask := byte(n)
+ if ('U'^mask)&7 >= 5 {
+ xorProfiles = append(xorProfiles, wireChoice{mode: WireXOR, mask: mask})
+ }
+ }
+ }
+
+ paddingRange := []uint16{0, 1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 768, 1024, 1400, 2048, 4096}
+ makeCovered := func(n int, xor, clear bool) cover.Profile {
+ first := byte(n)
+ second := byte(n*197 + 101)
+ mask := byte(n*149 + 37)
+ return cover.Profile{
+ Enabled: true,
+ ID: uint16(first)<<8 | uint16(second),
+ Padding: paddingRange[n%len(paddingRange)],
+ HeaderMask: mask,
+ XOR: xor,
+ Clear: clear,
+ }
+ }
+
+ // New peers try the clear-payload profile first. The next candidates are
+ // legacy direct profiles, so an older server falls back immediately instead
+ // of screening the complete expanded profile range.
+ out := make([]wireChoice, 0, len(binaryProfiles)+len(xorProfiles)+1025)
+ if s.configured == WireAuto || s.configured == WireBinary {
+ profile := makeCovered(0, false, true)
+ out = append(out, wireChoice{mode: WireBinary, mask: profile.HeaderMask, cover: profile})
+ }
+ if s.configured == WireAuto || s.configured == WireBP {
+ profile := makeCovered(0, false, true)
+ out = append(out, wireChoice{mode: WireBP, mask: profile.HeaderMask, cover: profile})
+ }
+
+ // Interleave formats so neither family can consume the entire discovery
+ // window before the other one gets a chance.
+ for i := 0; i < len(binaryProfiles) || i < len(xorProfiles); i++ {
+ if i < len(binaryProfiles) {
+ out = append(out, binaryProfiles[i])
+ }
+ if i < len(xorProfiles) {
+ out = append(out, xorProfiles[i])
+ }
+ if i == 0 && s.configured == WireAuto {
+ out = append(out, wireChoice{mode: WireBP})
+ }
+ }
+ if s.configured == WireBP {
+ out = append(out, wireChoice{mode: WireBP})
+ }
+
+ // Covered profiles expand discovery beyond the one-byte direct formats
+ // without taking the Cartesian product (which would create thousands of
+ // connections). Across this distributed range each wire still exercises all
+ // 256 first bytes, all 256 frame masks, and every padding length repeatedly.
+ for n := 0; n < 256; n++ {
+ if s.configured == WireAuto || s.configured == WireBinary {
+ profile := makeCovered(n, false, false)
+ out = append(out, wireChoice{mode: WireBinary, mask: profile.HeaderMask, cover: profile})
+ if n != 0 {
+ profile = makeCovered(n, false, true)
+ out = append(out, wireChoice{mode: WireBinary, mask: profile.HeaderMask, cover: profile})
+ }
+ }
+ if s.configured == WireAuto || s.configured == WireBP {
+ if n != 0 {
+ profile := makeCovered(n, false, true)
+ out = append(out, wireChoice{mode: WireBP, mask: profile.HeaderMask, cover: profile})
+ }
+ }
+ if s.configured == WireAuto || s.configured == WireXOR {
+ profile := makeCovered(n, true, false)
+ out = append(out, wireChoice{mode: WireXOR, mask: profile.HeaderMask, cover: profile})
+ }
+ }
+ return out
+}
+
+// detectLocked validates candidates with real HTTP traffic through ip.dr2.site.
+// The default is one worker. Users may explicitly allow more workers, while the
+// launch delay still spaces new attempts globally to avoid a connection burst.
+func (s *wireSelector) detectLocked() (wireChoice, bool) {
+ candidates := s.profileCandidates()
+ if s.candidateOverride != nil {
+ candidates = s.candidateOverride
+ }
+ threads := s.probeThreads
+ if threads < 1 {
+ threads = 1
+ }
+ if threads > 16 {
+ threads = 16
+ }
+ delay := s.probeDelay
+ if delay <= 0 {
+ delay = time.Second
+ }
+ type result struct {
+ choice wireChoice
+ ok bool
+ }
+ results := make(chan result, threads)
+ next := 0
+ inflight := 0
+ completed := 0
+ started := time.Now()
+ var lastLaunch time.Time
+ for next < len(candidates) || inflight > 0 {
+ canLaunch := next < len(candidates) && inflight < threads
+ if canLaunch && (lastLaunch.IsZero() || time.Since(lastLaunch) >= delay) {
+ candidate := candidates[next]
+ next++
+ inflight++
+ lastLaunch = time.Now()
+ go func(choice wireChoice) {
+ validated := false
+ if s.probeOverride != nil {
+ validated = s.probeOverride(choice)
+ } else {
+ validated = s.probe(choice)
+ }
+ results <- result{choice: choice, ok: validated}
+ }(candidate)
+ continue
+ }
+
+ var got result
+ if canLaunch {
+ wait := delay - time.Since(lastLaunch)
+ timer := time.NewTimer(wait)
+ select {
+ case got = <-results:
+ if !timer.Stop() {
+ select {
+ case <-timer.C:
+ default:
+ }
+ }
+ case <-timer.C:
+ continue
+ }
+ } else {
+ got = <-results
+ }
+ inflight--
+ completed++
+ if got.ok {
+ fmt.Printf("wire probe: selected=%s completed=%d launched=%d elapsed=%s target=http://%s/ validated=true threads=%d fixed_until_restart=true\n", got.choice, completed, next, time.Since(started).Round(time.Millisecond), probeHost, threads)
+ return got.choice, true
+ }
+ if completed%32 == 0 {
+ fmt.Printf("wire probe: completed=%d/%d launched=%d elapsed=%s target=http://%s/ no validated profile yet\n", completed, len(candidates), next, time.Since(started).Round(time.Millisecond), probeHost)
+ }
+ }
+ fmt.Printf("wire probe: no profile validated through http://%s/ after %d candidates in %s; retrying later\n", probeHost, completed, time.Since(started).Round(time.Millisecond))
+ return wireChoice{}, false
}
// probe fetches probeHost through one wire and reports whether a well-formed
// HTTP status line came back.
-func (s *wireSelector) probe(mode string) bool {
- type result struct{ ok bool }
- done := make(chan result, 1)
-
- go func() {
- var (
- conn net.Conn
- err error
- )
- if mode == WireXOR {
- conn, err = xorchunk.Open(s.serverAddr, s.token, probeHost, probePort, s.xorOpts)
+func (s *wireSelector) probe(choice wireChoice) bool {
+ var (
+ conn net.Conn
+ err error
+ )
+ if choice.mode == WireXOR {
+ if choice.cover.Enabled {
+ conn, err = xorchunk.Open(s.serverAddr, s.token, probeHost, probePort, s.xorOpts.WithCoverProfile(choice.cover))
} else {
- conn, err = openChunkTunnel(s.serverAddr, s.token, probeHost, probePort, s.binOpts)
+ conn, err = xorchunk.Open(s.serverAddr, s.token, probeHost, probePort, s.xorOpts.WithHeaderMask(choice.mask))
}
- if err != nil {
- done <- result{false}
- return
- }
- defer conn.Close()
-
- request := "GET / HTTP/1.1\r\nHost: " + probeHost + "\r\nUser-Agent: dragontcp\r\nConnection: close\r\n\r\n"
- if _, err := conn.Write([]byte(request)); err != nil {
- done <- result{false}
- return
- }
- buf := make([]byte, 64)
- n, err := conn.Read(buf)
- if n <= 0 || (err != nil && n == 0) {
- done <- result{false}
- return
- }
- done <- result{strings.HasPrefix(string(buf[:n]), "HTTP/")}
- }()
-
- select {
- case r := <-done:
- return r.ok
- case <-time.After(probeTimeout):
- // The tunnel goroutine is left to unwind on its own; the wire simply
- // did not answer in time, which is all the caller needs to know.
+ } else if choice.mode == WireBP {
+ opts := s.binOpts
+ opts.headerMask = choice.mask
+ opts.coverProfile = choice.cover
+ opts.skipPathProbe = true
+ opts.minSize = 32
+ opts.startSize = 32
+ opts.maxSize = 32
+ conn, err = openBPTunnel(s.serverAddr, s.token, probeHost, probePort, opts)
+ } else {
+ opts := s.binOpts
+ opts.headerMask = choice.mask
+ opts.coverProfile = choice.cover
+ opts.skipPathProbe = true
+ opts.minSize = 32
+ opts.startSize = 32
+ opts.maxSize = 32
+ conn, err = openChunkTunnel(s.serverAddr, s.token, probeHost, probePort, opts)
+ }
+ if err != nil {
return false
}
+ defer conn.Close()
+ _ = conn.SetDeadline(time.Now().Add(probeTimeout))
+
+ request := "GET / HTTP/1.1\r\nHost: " + probeHost + "\r\nUser-Agent: dragontcp\r\nConnection: close\r\n\r\n"
+ if _, err := conn.Write([]byte(request)); err != nil {
+ return false
+ }
+ statusLine, err := bufio.NewReader(conn).ReadString('\n')
+ return err == nil && strings.HasPrefix(statusLine, "HTTP/")
}
diff --git a/core/cmd/dragontcp-client/wireselect_test.go b/core/cmd/dragontcp-client/wireselect_test.go
new file mode 100644
index 0000000..cd3cc8f
--- /dev/null
+++ b/core/cmd/dragontcp-client/wireselect_test.go
@@ -0,0 +1,171 @@
+package main
+
+import (
+ "sync/atomic"
+ "testing"
+ "time"
+)
+
+func TestProfileCandidatesCoverBothWireFamilies(t *testing.T) {
+ selector := &wireSelector{configured: WireAuto}
+ candidates := selector.profileCandidates()
+ binaryCount, bpCount, xorCount := 0, 0, 0
+ seen := make(map[wireChoice]bool, len(candidates))
+ firstBytes := make(map[byte]bool, 256)
+ coveredMasks := map[string]map[byte]bool{WireBinary: {}, WireBP: {}, WireXOR: {}}
+ coveredPadding := map[string]map[uint16]bool{WireBinary: {}, WireBP: {}, WireXOR: {}}
+ for _, candidate := range candidates {
+ if seen[candidate] {
+ t.Fatalf("duplicate candidate: %s", candidate)
+ }
+ seen[candidate] = true
+ if candidate.cover.Enabled {
+ coveredMasks[candidate.mode][candidate.mask] = true
+ coveredPadding[candidate.mode][candidate.cover.Padding] = true
+ }
+ switch candidate.mode {
+ case WireBinary:
+ binaryCount++
+ if !candidate.cover.Enabled && candidate.mask&7 != 0 {
+ t.Fatalf("ambiguous binary mask: %02x", candidate.mask)
+ }
+ if candidate.cover.Enabled {
+ firstBytes[byte(candidate.cover.ID>>8)] = true
+ } else {
+ for mode := byte(0); mode <= 4; mode++ {
+ firstBytes[mode^candidate.mask] = true
+ }
+ }
+ case WireXOR:
+ xorCount++
+ if !candidate.cover.Enabled && ('U'^candidate.mask)&7 < 5 {
+ t.Fatalf("ambiguous XOR mask: %02x", candidate.mask)
+ }
+ if candidate.cover.Enabled {
+ firstBytes[byte(candidate.cover.ID>>8)] = true
+ } else {
+ firstBytes['U'^candidate.mask] = true
+ }
+ case WireBP:
+ bpCount++
+ if candidate.cover.Enabled && !candidate.cover.Clear {
+ t.Fatalf("covered BP profile must use clear payloads: %s", candidate)
+ }
+ if !candidate.cover.Enabled && candidate.mask != 0 {
+ t.Fatalf("direct BP profile must keep a clear header: %s", candidate)
+ }
+ default:
+ t.Fatalf("unknown candidate: %s", candidate)
+ }
+ }
+ if binaryCount != 544 || bpCount != 257 || xorCount != 352 {
+ t.Fatalf("profiles B=%d BP=%d X=%d, want B=544 BP=257 X=352", binaryCount, bpCount, xorCount)
+ }
+ if len(firstBytes) != 256 {
+ t.Fatalf("profiles cover %d first-byte values, want 256", len(firstBytes))
+ }
+ for _, mode := range []string{WireBinary, WireBP, WireXOR} {
+ if len(coveredMasks[mode]) != 256 {
+ t.Fatalf("mode %s covers %d masks, want 256", mode, len(coveredMasks[mode]))
+ }
+ if len(coveredPadding[mode]) != 16 {
+ t.Fatalf("mode %s covers %d padding lengths, want 16", mode, len(coveredPadding[mode]))
+ }
+ }
+}
+
+func TestManualWireStillDiscoversAllProfilesForThatFamily(t *testing.T) {
+ for _, tc := range []struct {
+ mode string
+ want int
+ }{{WireBinary, 544}, {WireBP, 257}, {WireXOR, 352}} {
+ selector := &wireSelector{configured: tc.mode}
+ candidates := selector.profileCandidates()
+ if len(candidates) != tc.want {
+ t.Fatalf("mode %s profiles=%d, want %d", tc.mode, len(candidates), tc.want)
+ }
+ for _, candidate := range candidates {
+ if candidate.mode != tc.mode {
+ t.Fatalf("mode %s included %s", tc.mode, candidate)
+ }
+ }
+ }
+}
+
+func TestClearProfilesAreTriedBeforeLegacyFallbacks(t *testing.T) {
+ for _, mode := range []string{WireBinary, WireBP} {
+ candidates := (&wireSelector{configured: mode}).profileCandidates()
+ if len(candidates) < 2 || !candidates[0].cover.Clear {
+ t.Fatalf("mode %s does not prefer a clear profile", mode)
+ }
+ if candidates[1].cover.Enabled {
+ t.Fatalf("mode %s does not fall back immediately to a legacy direct profile", mode)
+ }
+ }
+}
+
+func measureDiscoveryConcurrency(t *testing.T, threads int) int32 {
+ t.Helper()
+ candidates := make([]wireChoice, 24)
+ for i := range candidates {
+ candidates[i] = wireChoice{mode: WireBinary, mask: byte(i * 8)}
+ }
+ var active atomic.Int32
+ var maximum atomic.Int32
+ var calls atomic.Int32
+ selector := &wireSelector{
+ configured: WireAuto,
+ candidateOverride: candidates,
+ probeThreads: threads,
+ probeDelay: time.Nanosecond,
+ probeOverride: func(wireChoice) bool {
+ current := active.Add(1)
+ for {
+ old := maximum.Load()
+ if current <= old || maximum.CompareAndSwap(old, current) {
+ break
+ }
+ }
+ calls.Add(1)
+ // Keep attempts alive long enough for the globally spaced scheduler
+ // to fill every configured worker reliably on slower CI runners.
+ time.Sleep(20 * time.Millisecond)
+ active.Add(-1)
+ return false
+ },
+ }
+
+ if _, ok := selector.detectLocked(); ok {
+ t.Fatal("unexpected working profile")
+ }
+ if calls.Load() != int32(len(candidates)) {
+ t.Fatalf("screened=%d, want %d", calls.Load(), len(candidates))
+ }
+ return maximum.Load()
+}
+
+func TestDiscoveryDefaultsToOneWorker(t *testing.T) {
+ if maximum := measureDiscoveryConcurrency(t, 0); maximum != 1 {
+ t.Fatalf("maximum concurrent probes=%d, want 1", maximum)
+ }
+}
+
+func TestDiscoveryHonorsConfiguredWorkers(t *testing.T) {
+ if maximum := measureDiscoveryConcurrency(t, 4); maximum != 4 {
+ t.Fatalf("maximum concurrent probes=%d, want 4", maximum)
+ }
+}
+
+func TestFailedManualDiscoveryKeepsPinnedWire(t *testing.T) {
+ for _, mode := range []string{WireBinary, WireBP, WireXOR} {
+ selector := &wireSelector{
+ configured: mode,
+ candidateOverride: []wireChoice{{mode: mode}},
+ probeDelay: time.Nanosecond,
+ probeOverride: func(wireChoice) bool { return false },
+ }
+ if choice := selector.mode(); choice.mode != mode {
+ t.Fatalf("configured=%s fallback=%s", mode, choice.mode)
+ }
+ }
+}
diff --git a/core/cmd/dragontcp-server/bhttp.go b/core/cmd/dragontcp-server/bhttp.go
new file mode 100644
index 0000000..f8ffd73
--- /dev/null
+++ b/core/cmd/dragontcp-server/bhttp.go
@@ -0,0 +1,653 @@
+package main
+
+import (
+ "bufio"
+ "bytes"
+ "context"
+ "encoding/binary"
+ "fmt"
+ "io"
+ "net"
+ "sync"
+ "time"
+
+ "dragontcp/internal/wire"
+)
+
+const (
+ bhttpModeProbe byte = 0
+ bhttpModeUpload byte = 1
+ bhttpModeDownload byte = 2
+ bhttpModeBatchDownload byte = 3
+ bhttpModeACK byte = 4
+ bhttpProbeVersion byte = 1
+ bhttpRequestHeaderSize = 29
+)
+
+var bhttpProbeMagic = [4]byte{'B', 'H', 'P', '1'}
+var bhttpOpenMagic = [4]byte{'D', 'O', 'P', '1'}
+var bpCloseMagic = [4]byte{'D', 'C', 'L', '1'}
+
+// bhttpSession intentionally models only the transport/session behavior that
+// is observable in bhttp_remote_test.py. The supplied client test contains no
+// destination-selection handshake, so uploads are acknowledged and counted but
+// are not forwarded to an invented target.
+type bhttpSession struct {
+ mu sync.Mutex
+ lastSeen time.Time
+ uploaded uint64
+ acked uint64
+ stream *streamSession
+}
+
+func (s *bhttpSession) touch() {
+ s.mu.Lock()
+ s.lastSeen = time.Now()
+ s.mu.Unlock()
+}
+
+type bhttpSessionManager struct {
+ mu sync.RWMutex
+ sessions map[string]*bhttpSession
+ timeout time.Duration
+ max int
+}
+
+func newBHTTPSessionManager(timeout time.Duration, max int) *bhttpSessionManager {
+ if timeout <= 0 {
+ timeout = 2 * time.Minute
+ }
+ if max < 1 {
+ max = 1
+ }
+ m := &bhttpSessionManager{
+ sessions: make(map[string]*bhttpSession),
+ timeout: timeout,
+ max: max,
+ }
+ go m.cleanupLoop()
+ return m
+}
+
+func (m *bhttpSessionManager) get(sid wire.SessionID) *bhttpSession {
+ m.mu.RLock()
+ s := m.sessions[sidKey(sid)]
+ m.mu.RUnlock()
+ if s != nil {
+ s.touch()
+ }
+ return s
+}
+
+func (m *bhttpSessionManager) register(sid wire.SessionID) bool {
+ key := sidKey(sid)
+ m.mu.Lock()
+ if old := m.sessions[key]; old != nil {
+ m.mu.Unlock()
+ old.touch()
+ return true
+ }
+ if len(m.sessions) >= m.max {
+ m.mu.Unlock()
+ return false
+ }
+ m.sessions[key] = &bhttpSession{lastSeen: time.Now()}
+ m.mu.Unlock()
+ return true
+}
+
+func (m *bhttpSessionManager) remove(sid wire.SessionID) bool {
+ key := sidKey(sid)
+ m.mu.Lock()
+ session := m.sessions[key]
+ delete(m.sessions, key)
+ m.mu.Unlock()
+ if session == nil {
+ return false
+ }
+ session.mu.Lock()
+ stream := session.stream
+ session.stream = nil
+ session.mu.Unlock()
+ if stream != nil {
+ stream.close()
+ }
+ return true
+}
+
+func (m *bhttpSessionManager) cleanupLoop() {
+ ticker := time.NewTicker(30 * time.Second)
+ defer ticker.Stop()
+ for now := range ticker.C {
+ cutoff := now.Add(-m.timeout)
+ var closing []*streamSession
+ m.mu.Lock()
+ for key, session := range m.sessions {
+ session.mu.Lock()
+ stale := session.lastSeen.Before(cutoff)
+ stream := session.stream
+ session.mu.Unlock()
+ if stale {
+ delete(m.sessions, key)
+ if stream != nil {
+ closing = append(closing, stream)
+ }
+ }
+ }
+ m.mu.Unlock()
+ for _, stream := range closing {
+ stream.close()
+ }
+ }
+}
+
+type bhttpRequest struct {
+ mode byte
+ session wire.SessionID
+ seq uint64
+ value uint32
+ payload []byte
+ headerMask byte
+ clear bool
+}
+
+type binaryHeader struct {
+ mode byte
+ session wire.SessionID
+ seq uint64
+ length uint32
+}
+
+func peekBinaryHeader(r *bufio.Reader, headerMask byte) (binaryHeader, error) {
+ var out binaryHeader
+ header, err := r.Peek(bhttpRequestHeaderSize)
+ if err != nil {
+ return out, err
+ }
+ out.mode = header[0] ^ headerMask
+ copy(out.session[:], header[1:17])
+ out.seq = binary.BigEndian.Uint64(header[17:25])
+ out.length = binary.BigEndian.Uint32(header[25:29])
+ return out, nil
+}
+
+func readBHTTPRequest(r *bufio.Reader, headerMask byte, clear bool) (bhttpRequest, error) {
+ var req bhttpRequest
+ var header [bhttpRequestHeaderSize]byte
+ if _, err := io.ReadFull(r, header[:]); err != nil {
+ return req, err
+ }
+ req.mode = header[0] ^ headerMask
+ req.headerMask = headerMask
+ req.clear = clear
+ if req.mode > bhttpModeACK {
+ return req, fmt.Errorf("unknown BP mode")
+ }
+ copy(req.session[:], header[1:17])
+ req.seq = binary.BigEndian.Uint64(header[17:25])
+ req.value = binary.BigEndian.Uint32(header[25:29])
+
+ // BHTTP mode 2 overloads the normal body-length field as a download-size
+ // hint and sends no payload bytes after the 29-byte header.
+ if req.mode == bhttpModeDownload {
+ return req, nil
+ }
+ if req.value > wire.MaxPayload {
+ return req, fmt.Errorf("BP payload too large")
+ }
+ if req.value > 0 {
+ req.payload = make([]byte, int(req.value))
+ if _, err := io.ReadFull(r, req.payload); err != nil {
+ return req, err
+ }
+ if !clear {
+ wire.MaskInPlace(req.payload, req.session, req.mode, req.seq, false)
+ }
+ }
+ return req, nil
+}
+
+func parseBHTTPProbe(payload []byte) (byte, int, error) {
+ if len(payload) < 10 || !bytes.Equal(payload[:4], bhttpProbeMagic[:]) || payload[4] != bhttpProbeVersion {
+ return 0, 0, fmt.Errorf("bad BP probe")
+ }
+ submode := payload[5]
+ if submode > bhttpModeACK {
+ return 0, 0, fmt.Errorf("unknown BP probe submode")
+ }
+ param := int(binary.BigEndian.Uint32(payload[6:10]))
+ want := 10
+ if submode == bhttpModeUpload && param >= 10 {
+ want = param
+ }
+ if len(payload) != want {
+ return 0, 0, fmt.Errorf("bad BP probe length")
+ }
+ for i := 10; i < len(payload); i++ {
+ if payload[i] != byte(i*31) {
+ return 0, 0, fmt.Errorf("bad BP probe pattern")
+ }
+ }
+ return submode, param, nil
+}
+
+func makeBHTTPProbe(submode byte, param int) []byte {
+ total := 10
+ if submode == bhttpModeDownload && param > total {
+ total = param
+ }
+ out := make([]byte, total)
+ copy(out[:4], bhttpProbeMagic[:])
+ out[4] = bhttpProbeVersion
+ out[5] = submode
+ binary.BigEndian.PutUint32(out[6:10], uint32(param))
+ for i := 10; i < len(out); i++ {
+ out[i] = byte(i * 31)
+ }
+ return out
+}
+
+func writeBHTTPError(conn net.Conn, message string) error {
+ return wire.WriteResponse(conn, wire.StatusError, []byte(message))
+}
+
+func writeBHTTPMasked(conn net.Conn, status byte, body []byte, req bhttpRequest) error {
+ return wire.WriteMaskedResponseProfileEncoding(conn, status, body, req.session, req.mode, req.seq, req.headerMask, req.clear)
+}
+
+func writeBHTTPData(conn net.Conn, req bhttpRequest, data []byte) error {
+ // Build and mask the complete response once. The generic two-step path
+ // first built a BP body and then copied it into another framed packet,
+ // temporarily allocating roughly twice the download size.
+ if req.clear {
+ var header [wire.ResponseHeaderSize]byte
+ header[0] = wire.StatusData ^ req.headerMask
+ binary.BigEndian.PutUint32(header[1:5], uint32(4+len(data)))
+ var length [4]byte
+ binary.BigEndian.PutUint32(length[:], uint32(len(data)))
+ buffers := net.Buffers{header[:], length[:], data}
+ _, err := buffers.WriteTo(conn)
+ return err
+ }
+ packet := make([]byte, wire.ResponseHeaderSize+4+len(data))
+ packet[0] = wire.StatusData ^ req.headerMask
+ binary.BigEndian.PutUint32(packet[1:5], uint32(4+len(data)))
+ binary.BigEndian.PutUint32(packet[5:9], uint32(len(data)))
+ copy(packet[9:], data)
+ wire.MaskInPlace(packet[5:], req.session, req.mode, req.seq, true)
+ for len(packet) > 0 {
+ n, err := conn.Write(packet)
+ if err != nil {
+ return err
+ }
+ if n <= 0 {
+ return io.ErrShortWrite
+ }
+ packet = packet[n:]
+ }
+ return nil
+}
+
+type bhttpServerContext struct {
+ sessions *bhttpSessionManager
+ token string
+ allowPrivate bool
+ cache *dnsCache
+ tcpBuffer int
+ maxChunk int
+ maxBuffer int
+ pollWait time.Duration
+ debug *serverDebug
+}
+
+func processBHTTPRequest(conn net.Conn, req bhttpRequest, ctx *bhttpServerContext) error {
+ sessions := ctx.sessions
+ maxChunk := ctx.maxChunk
+ switch req.mode {
+ case bhttpModeProbe:
+ submode, param, err := parseBHTTPProbe(req.payload)
+ if err != nil {
+ return writeBHTTPError(conn, err.Error())
+ }
+ if submode == bhttpModeUpload && len(req.payload) > maxChunk {
+ return writeBHTTPError(conn, "probe too large")
+ }
+ if submode == bhttpModeDownload && (param < 0 || param > maxChunk) {
+ return writeBHTTPError(conn, "probe too large")
+ }
+ count := 1
+ if submode == bhttpModeACK {
+ count = param
+ if count < 1 {
+ count = 1
+ }
+ if count > 256 {
+ count = 256
+ }
+ }
+ body := makeBHTTPProbe(submode, param)
+ for i := 0; i < count; i++ {
+ // The reference client decrypts every batch echo with the original
+ // request sequence, rather than incrementing it per response.
+ if err := writeBHTTPMasked(conn, wire.StatusOK, body, req); err != nil {
+ return err
+ }
+ }
+ return nil
+
+ case bhttpModeUpload:
+ if req.seq == 0 && len(req.payload) == 0 {
+ if !sessions.register(req.session) {
+ return writeBHTTPError(conn, "session limit reached")
+ }
+ return wire.WriteResponse(conn, wire.StatusOK, nil)
+ }
+ session := sessions.get(req.session)
+ if session == nil {
+ return writeBHTTPError(conn, "unknown session")
+ }
+ if len(req.payload) > maxChunk {
+ return writeBHTTPError(conn, "upload too large")
+ }
+ if req.seq == 1 && len(req.payload) >= len(bhttpOpenMagic) && bytes.Equal(req.payload[:len(bhttpOpenMagic)], bhttpOpenMagic[:]) {
+ supplied, host, port, err := parseOpen(req.payload[len(bhttpOpenMagic):])
+ if err != nil {
+ return writeBHTTPError(conn, err.Error())
+ }
+ if !tokenEqual(supplied, ctx.token) {
+ return writeBHTTPError(conn, "authentication failed")
+ }
+ session.mu.Lock()
+ alreadyOpen := session.stream != nil
+ session.mu.Unlock()
+ if alreadyOpen {
+ return wire.WriteResponse(conn, wire.StatusOK, nil)
+ }
+ dialCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
+ target, err := dialTarget(dialCtx, host, port, ctx.allowPrivate, ctx.cache, ctx.tcpBuffer)
+ cancel()
+ if err != nil {
+ return writeBHTTPError(conn, err.Error())
+ }
+ stream := newStreamSession(req.session, target, fmt.Sprintf("%s:%d", host, port), maxChunk, ctx.maxBuffer, ctx.debug)
+ session.mu.Lock()
+ if session.stream == nil {
+ session.stream = stream
+ session.lastSeen = time.Now()
+ stream = nil
+ }
+ session.mu.Unlock()
+ if stream != nil {
+ stream.close()
+ }
+ if ctx.debug != nil && ctx.debug.enabled {
+ ctx.debug.logf("BP OPEN sid=%x target=%s:%d", req.session[:4], host, port)
+ }
+ return wire.WriteResponse(conn, wire.StatusOK, nil)
+ }
+
+ session.mu.Lock()
+ stream := session.stream
+ session.mu.Unlock()
+ if stream != nil {
+ if req.seq < 2 {
+ return writeBHTTPError(conn, "bad upload sequence")
+ }
+ if err := stream.upload(req.seq-2, req.payload); err != nil {
+ return writeBHTTPError(conn, err.Error())
+ }
+ }
+ session.mu.Lock()
+ session.uploaded += uint64(len(req.payload))
+ session.lastSeen = time.Now()
+ session.mu.Unlock()
+ return wire.WriteResponse(conn, wire.StatusOK, nil)
+
+ case bhttpModeDownload:
+ session := sessions.get(req.session)
+ if session == nil {
+ return writeBHTTPError(conn, "unknown session")
+ }
+ session.mu.Lock()
+ stream := session.stream
+ session.mu.Unlock()
+ if stream == nil {
+ // The reference transport has no observable downstream producer.
+ return wire.WriteResponse(conn, wire.StatusOK, nil)
+ }
+ limit := int(req.value)
+ if limit < 1 {
+ limit = 1
+ }
+ if limit > maxChunk {
+ limit = maxChunk
+ }
+ data, status, err := stream.readAt(req.seq, limit, ctx.pollWait)
+ if err != nil {
+ return writeBHTTPError(conn, err.Error())
+ }
+ switch status {
+ case wire.StatusData:
+ return writeBHTTPData(conn, req, data)
+ case wire.StatusEOF:
+ return wire.WriteResponse(conn, wire.StatusEOF, nil)
+ default:
+ return wire.WriteResponse(conn, wire.StatusOK, nil)
+ }
+
+ case bhttpModeBatchDownload:
+ session := sessions.get(req.session)
+ if session == nil {
+ return writeBHTTPError(conn, "unknown session")
+ }
+ if len(req.payload) != 6 {
+ return writeBHTTPError(conn, "bad batch download request")
+ }
+ count := int(binary.BigEndian.Uint16(req.payload[4:6]))
+ limit := int(binary.BigEndian.Uint32(req.payload[:4]))
+ if limit < 1 {
+ limit = 1
+ }
+ if limit > maxChunk {
+ limit = maxChunk
+ }
+ if count < 1 {
+ count = 1
+ }
+ if count > 256 {
+ count = 256
+ }
+ session.mu.Lock()
+ stream := session.stream
+ session.mu.Unlock()
+ offset := req.seq
+ for i := 0; i < count; i++ {
+ if stream == nil {
+ if err := wire.WriteResponse(conn, wire.StatusOK, nil); err != nil {
+ return err
+ }
+ continue
+ }
+ wait := time.Duration(0)
+ if i == 0 {
+ wait = ctx.pollWait
+ }
+ data, status, err := stream.readAt(offset, limit, wait)
+ if err != nil {
+ return writeBHTTPError(conn, err.Error())
+ }
+ switch status {
+ case wire.StatusData:
+ if err := writeBHTTPData(conn, req, data); err != nil {
+ return err
+ }
+ offset += uint64(len(data))
+ case wire.StatusEOF:
+ if err := wire.WriteResponse(conn, wire.StatusEOF, nil); err != nil {
+ return err
+ }
+ default:
+ if err := wire.WriteResponse(conn, wire.StatusOK, nil); err != nil {
+ return err
+ }
+ }
+ }
+ return nil
+
+ case bhttpModeACK:
+ session := sessions.get(req.session)
+ if session == nil {
+ return writeBHTTPError(conn, "unknown session")
+ }
+ // Dragon's BP extension sends an explicit close marker. Reference BP
+ // clients continue to use an empty ACK, while Dragon clients release the
+ // target socket and buffered download data immediately instead of waiting
+ // for the idle-session reaper.
+ if bytes.Equal(req.payload, bpCloseMagic[:]) {
+ sessions.remove(req.session)
+ return wire.WriteResponse(conn, wire.StatusOK, nil)
+ }
+ session.mu.Lock()
+ if req.seq > session.acked {
+ session.acked = req.seq
+ }
+ session.lastSeen = time.Now()
+ stream := session.stream
+ session.mu.Unlock()
+ if stream != nil {
+ stream.ack(req.seq)
+ }
+ return wire.WriteResponse(conn, wire.StatusOK, nil)
+ }
+ return writeBHTTPError(conn, "unknown mode")
+}
+
+type binaryFlavor byte
+
+const (
+ binaryFlavorUnknown binaryFlavor = iota
+ binaryFlavorDragon
+ binaryFlavorBHTTP
+)
+
+func isBHTTPProbe(payload []byte) bool {
+ return len(payload) >= 4 && bytes.Equal(payload[:4], bhttpProbeMagic[:])
+}
+
+// handleBinary auto-detects the two protocols without changing the native B
+// header space. BHTTP is clear-header only; Dragon profiles and cover-prefaced
+// connections continue through the existing handler unchanged.
+func handleBinary(
+ conn net.Conn,
+ headerMask byte,
+ clearPayload bool,
+ token string,
+ allowPrivate bool,
+ cache *dnsCache,
+ tcpBuffer int,
+ manager *streamManager,
+ bhttp *bhttpSessionManager,
+ chunkMax int,
+ bufferBytes int,
+ pollWait time.Duration,
+ debug *serverDebug,
+) {
+ reader := bufio.NewReader(conn)
+ bhttpContext := &bhttpServerContext{
+ sessions: bhttp,
+ token: token,
+ allowPrivate: allowPrivate,
+ cache: cache,
+ tcpBuffer: tcpBuffer,
+ maxChunk: chunkMax,
+ maxBuffer: bufferBytes,
+ pollWait: pollWait,
+ debug: debug,
+ }
+ flavor := binaryFlavorUnknown
+ deadline := newIdleDeadline(conn, 30*time.Second)
+ for {
+ if deadline.refresh() != nil {
+ return
+ }
+ header, err := peekBinaryHeader(reader, headerMask)
+ if err != nil {
+ return
+ }
+
+ if flavor == binaryFlavorUnknown {
+ switch header.mode {
+ case bhttpModeProbe:
+ // Probe framing is shared, so consume it once and use its magic
+ // to select BHP1 or DTP2 without losing any bytes.
+ req, err := wire.ReadRequestProfileEncoding(reader, headerMask, clearPayload)
+ if err != nil {
+ return
+ }
+ if isBHTTPProbe(req.Payload) {
+ flavor = binaryFlavorBHTTP
+ breq := bhttpRequest{mode: req.Mode, session: req.Session, seq: req.Seq, value: uint32(len(req.Payload)), payload: req.Payload, headerMask: headerMask, clear: clearPayload}
+ if processBHTTPRequest(conn, breq, bhttpContext) != nil {
+ return
+ }
+ continue
+ }
+ flavor = binaryFlavorDragon
+ if processWireRequest(conn, req, token, allowPrivate, cache, tcpBuffer, manager, chunkMax, bufferBytes, pollWait, debug) != nil {
+ return
+ }
+ continue
+
+ case bhttpModeUpload:
+ if bhttp.get(header.session) != nil || (header.seq == 0 && header.length == 0) {
+ flavor = binaryFlavorBHTTP
+ } else {
+ flavor = binaryFlavorDragon
+ }
+ case bhttpModeDownload:
+ if bhttp.get(header.session) != nil {
+ flavor = binaryFlavorBHTTP
+ } else if manager.get(header.session) != nil {
+ flavor = binaryFlavorDragon
+ } else {
+ // The BHTTP unknown-session test sends only a header whose
+ // length field is a hint. Consume no nonexistent body.
+ if _, err := readBHTTPRequest(reader, headerMask, clearPayload); err == nil {
+ _ = writeBHTTPError(conn, "unknown session")
+ }
+ return
+ }
+ case bhttpModeBatchDownload:
+ if bhttp.get(header.session) != nil || header.length == 6 {
+ flavor = binaryFlavorBHTTP
+ } else {
+ flavor = binaryFlavorDragon
+ }
+ case bhttpModeACK:
+ if bhttp.get(header.session) != nil {
+ flavor = binaryFlavorBHTTP
+ } else {
+ flavor = binaryFlavorDragon
+ }
+ default:
+ return
+ }
+ }
+
+ if flavor == binaryFlavorBHTTP {
+ req, err := readBHTTPRequest(reader, headerMask, clearPayload)
+ if err != nil || processBHTTPRequest(conn, req, bhttpContext) != nil {
+ return
+ }
+ continue
+ }
+
+ req, err := wire.ReadRequestProfileEncoding(reader, headerMask, clearPayload)
+ if err != nil {
+ return
+ }
+ if processWireRequest(conn, req, token, allowPrivate, cache, tcpBuffer, manager, chunkMax, bufferBytes, pollWait, debug) != nil {
+ return
+ }
+ }
+}
diff --git a/core/cmd/dragontcp-server/bhttp_test.go b/core/cmd/dragontcp-server/bhttp_test.go
new file mode 100644
index 0000000..1ff595a
--- /dev/null
+++ b/core/cmd/dragontcp-server/bhttp_test.go
@@ -0,0 +1,301 @@
+package main
+
+import (
+ "bytes"
+ "encoding/binary"
+ "io"
+ "net"
+ "testing"
+ "time"
+
+ "dragontcp/internal/cover"
+ "dragontcp/internal/wire"
+)
+
+func writeBHTTPTestRequest(w io.Writer, mode byte, sid wire.SessionID, seq uint64, payload []byte, downloadHint uint32) error {
+ n := uint32(len(payload))
+ if mode == bhttpModeDownload {
+ n = downloadHint
+ payload = nil
+ }
+ packet := make([]byte, bhttpRequestHeaderSize+len(payload))
+ packet[0] = mode
+ copy(packet[1:17], sid[:])
+ binary.BigEndian.PutUint64(packet[17:25], seq)
+ binary.BigEndian.PutUint32(packet[25:29], n)
+ copy(packet[29:], payload)
+ wire.MaskInPlace(packet[29:], sid, mode, seq, false)
+ _, err := w.Write(packet)
+ return err
+}
+
+func readBHTTPTestResponse(r io.Reader, sid wire.SessionID, mode byte, seq uint64) (byte, []byte, error) {
+ status, body, err := wire.ReadResponse(r)
+ if err == nil && status != wire.StatusError {
+ wire.MaskInPlace(body, sid, mode, seq, true)
+ }
+ return status, body, err
+}
+
+func startBHTTPTestServer(t *testing.T, sessions *bhttpSessionManager) (net.Conn, <-chan struct{}) {
+ t.Helper()
+ server, client := net.Pipe()
+ done := make(chan struct{})
+ go func() {
+ defer close(done)
+ defer server.Close()
+ handleBinary(
+ server,
+ 0,
+ false,
+ "",
+ false,
+ newDNSCache(time.Minute, 16),
+ 0,
+ newStreamManager(time.Minute, nil),
+ sessions,
+ 1024*1024,
+ 1024*1024,
+ 10*time.Millisecond,
+ nil,
+ )
+ }()
+ return client, done
+}
+
+func TestBHTTPReferenceSessionStack(t *testing.T) {
+ sessions := newBHTTPSessionManager(time.Minute, 32)
+ client, done := startBHTTPTestServer(t, sessions)
+ defer func() {
+ client.Close()
+ <-done
+ }()
+ _ = client.SetDeadline(time.Now().Add(2 * time.Second))
+
+ var sid wire.SessionID
+ for i := range sid {
+ sid[i] = byte(i + 1)
+ }
+
+ if err := writeBHTTPTestRequest(client, bhttpModeUpload, sid, 0, nil, 0); err != nil {
+ t.Fatal(err)
+ }
+ if status, _, err := readBHTTPTestResponse(client, sid, bhttpModeUpload, 0); err != nil || status != wire.StatusOK {
+ t.Fatalf("registration status=%d err=%v", status, err)
+ }
+
+ if err := writeBHTTPTestRequest(client, bhttpModeUpload, sid, 1, []byte("Hello BHTTP"), 0); err != nil {
+ t.Fatal(err)
+ }
+ if status, _, err := readBHTTPTestResponse(client, sid, bhttpModeUpload, 1); err != nil || status != wire.StatusOK {
+ t.Fatalf("upload status=%d err=%v", status, err)
+ }
+
+ // The size is in the header but no 1,350-byte body follows. This is the
+ // framing difference that made the native Dragon parser wait forever.
+ if err := writeBHTTPTestRequest(client, bhttpModeDownload, sid, 0, nil, 1350); err != nil {
+ t.Fatal(err)
+ }
+ if status, _, err := readBHTTPTestResponse(client, sid, bhttpModeDownload, 0); err != nil || status != wire.StatusOK {
+ t.Fatalf("download status=%d err=%v", status, err)
+ }
+
+ batch := make([]byte, 6)
+ binary.BigEndian.PutUint32(batch[:4], 1350)
+ binary.BigEndian.PutUint16(batch[4:], 2)
+ if err := writeBHTTPTestRequest(client, bhttpModeBatchDownload, sid, 0, batch, 0); err != nil {
+ t.Fatal(err)
+ }
+ for i := 0; i < 2; i++ {
+ if status, _, err := readBHTTPTestResponse(client, sid, bhttpModeBatchDownload, 0); err != nil || status != wire.StatusOK {
+ t.Fatalf("batch response %d status=%d err=%v", i, status, err)
+ }
+ }
+
+ if err := writeBHTTPTestRequest(client, bhttpModeACK, sid, 5, nil, 0); err != nil {
+ t.Fatal(err)
+ }
+ if status, _, err := readBHTTPTestResponse(client, sid, bhttpModeACK, 5); err != nil || status != wire.StatusOK {
+ t.Fatalf("ack status=%d err=%v", status, err)
+ }
+}
+
+func TestBPExplicitCloseRemovesSession(t *testing.T) {
+ sessions := newBHTTPSessionManager(time.Minute, 32)
+ client, done := startBHTTPTestServer(t, sessions)
+ defer func() {
+ client.Close()
+ <-done
+ }()
+ _ = client.SetDeadline(time.Now().Add(2 * time.Second))
+
+ var sid wire.SessionID
+ copy(sid[:], []byte("close-session-01"))
+ if err := writeBHTTPTestRequest(client, bhttpModeUpload, sid, 0, nil, 0); err != nil {
+ t.Fatal(err)
+ }
+ if status, _, err := readBHTTPTestResponse(client, sid, bhttpModeUpload, 0); err != nil || status != wire.StatusOK {
+ t.Fatalf("registration status=%d err=%v", status, err)
+ }
+ if sessions.get(sid) == nil {
+ t.Fatal("registered session is missing")
+ }
+
+ if err := writeBHTTPTestRequest(client, bhttpModeACK, sid, 0, bpCloseMagic[:], 0); err != nil {
+ t.Fatal(err)
+ }
+ if status, _, err := readBHTTPTestResponse(client, sid, bhttpModeACK, 0); err != nil || status != wire.StatusOK {
+ t.Fatalf("close status=%d err=%v", status, err)
+ }
+ if sessions.get(sid) != nil {
+ t.Fatal("explicit close retained the session")
+ }
+}
+
+func TestBHTTPReferenceProbeAndBatchEcho(t *testing.T) {
+ sessions := newBHTTPSessionManager(time.Minute, 32)
+ client, done := startBHTTPTestServer(t, sessions)
+ defer func() {
+ client.Close()
+ <-done
+ }()
+ _ = client.SetDeadline(time.Now().Add(2 * time.Second))
+
+ var sid wire.SessionID
+ copy(sid[:], []byte("probe-session-01"))
+ payload := make([]byte, 10)
+ copy(payload[:4], []byte("BHP1"))
+ payload[4] = 1
+ payload[5] = bhttpModeDownload
+ binary.BigEndian.PutUint32(payload[6:], 512)
+ if err := writeBHTTPTestRequest(client, bhttpModeProbe, sid, 0, payload, 0); err != nil {
+ t.Fatal(err)
+ }
+ status, body, err := readBHTTPTestResponse(client, sid, bhttpModeProbe, 0)
+ if err != nil || status != wire.StatusOK || len(body) != 512 || !bytes.Equal(body[:10], payload) {
+ t.Fatalf("download probe status=%d len=%d err=%v", status, len(body), err)
+ }
+ for i := 10; i < len(body); i++ {
+ if body[i] != byte(i*31) {
+ t.Fatalf("probe pattern byte %d=%02x", i, body[i])
+ }
+ }
+
+ payload[5] = bhttpModeACK
+ binary.BigEndian.PutUint32(payload[6:], 3)
+ if err := writeBHTTPTestRequest(client, bhttpModeProbe, sid, 0, payload, 0); err != nil {
+ t.Fatal(err)
+ }
+ for i := 0; i < 3; i++ {
+ status, body, err := readBHTTPTestResponse(client, sid, bhttpModeProbe, 0)
+ if err != nil || status != wire.StatusOK || !bytes.Equal(body, payload) {
+ t.Fatalf("batch probe %d status=%d body=%x err=%v", i, status, body, err)
+ }
+ }
+}
+
+func TestBHTTPUnknownSessionDownloadHasNoBody(t *testing.T) {
+ sessions := newBHTTPSessionManager(time.Minute, 32)
+ client, done := startBHTTPTestServer(t, sessions)
+ defer func() {
+ client.Close()
+ <-done
+ }()
+ _ = client.SetDeadline(time.Now().Add(2 * time.Second))
+
+ var sid wire.SessionID
+ copy(sid[:], []byte("unknown-session!"))
+ if err := writeBHTTPTestRequest(client, bhttpModeDownload, sid, 0, nil, 1350); err != nil {
+ t.Fatal(err)
+ }
+ status, _, err := wire.ReadResponse(client)
+ if err != nil || status == wire.StatusOK || status == wire.StatusData {
+ t.Fatalf("unknown session status=%d err=%v", status, err)
+ }
+}
+
+func TestBinaryAutoDetectionKeepsNativeDragonProbe(t *testing.T) {
+ sessions := newBHTTPSessionManager(time.Minute, 32)
+ client, done := startBHTTPTestServer(t, sessions)
+ defer func() {
+ client.Close()
+ <-done
+ }()
+ _ = client.SetDeadline(time.Now().Add(2 * time.Second))
+
+ var sid wire.SessionID
+ payload := make([]byte, 11)
+ copy(payload[:4], wire.ProbeMagic[:])
+ payload[4] = wire.ProbeKeepalive
+ if err := wire.WriteRequest(client, wire.ModeProbe, sid, 1, payload); err != nil {
+ t.Fatal(err)
+ }
+ status, _, err := wire.ReadResponse(client)
+ if err != nil || status != wire.StatusOK {
+ t.Fatalf("native probe status=%d err=%v", status, err)
+ }
+}
+
+func TestClearCoveredBinaryAndBPProfiles(t *testing.T) {
+ for _, bp := range []bool{false, true} {
+ t.Run(map[bool]string{false: "B", true: "BP"}[bp], func(t *testing.T) {
+ server, client := net.Pipe()
+ profile := cover.Profile{Enabled: true, ID: 0x8173, Padding: 32, HeaderMask: 0x9b, Clear: true}
+ done := make(chan struct{})
+ go func() {
+ defer close(done)
+ defer server.Close()
+ profiled, isXOR, mask, err := sniffWire(server)
+ if err != nil || isXOR {
+ return
+ }
+ handleBinary(
+ profiled, mask, true, "", false,
+ newDNSCache(time.Minute, 16), 0,
+ newStreamManager(time.Minute, nil),
+ newBHTTPSessionManager(time.Minute, 32),
+ 1024*1024, 1024*1024, 10*time.Millisecond, nil,
+ )
+ }()
+ defer func() {
+ client.Close()
+ <-done
+ }()
+ _ = client.SetDeadline(time.Now().Add(2 * time.Second))
+ if err := cover.WritePreface(client, profile); err != nil {
+ t.Fatal(err)
+ }
+
+ var sid wire.SessionID
+ copy(sid[:], []byte("clear-profile-01"))
+ if bp {
+ payload := makeBHTTPProbe(bhttpModeDownload, 256)[:10]
+ packet := make([]byte, bhttpRequestHeaderSize+len(payload))
+ packet[0] = bhttpModeProbe ^ profile.HeaderMask
+ copy(packet[1:17], sid[:])
+ binary.BigEndian.PutUint32(packet[25:29], uint32(len(payload)))
+ copy(packet[29:], payload)
+ if _, err := client.Write(packet); err != nil {
+ t.Fatal(err)
+ }
+ status, body, err := wire.ReadResponseProfile(client, profile.HeaderMask)
+ if err != nil || status != wire.StatusOK || !bytes.Equal(body, makeBHTTPProbe(bhttpModeDownload, 256)) {
+ t.Fatalf("clear BP status=%d len=%d err=%v", status, len(body), err)
+ }
+ return
+ }
+
+ payload := make([]byte, 11)
+ copy(payload[:4], wire.ProbeMagic[:])
+ payload[4] = wire.ProbeDownload
+ binary.BigEndian.PutUint32(payload[7:11], 256)
+ if err := wire.WriteRequestProfileEncoding(client, wire.ModeProbe, sid, 7, payload, profile.HeaderMask, true); err != nil {
+ t.Fatal(err)
+ }
+ status, body, err := wire.ReadResponseProfile(client, profile.HeaderMask)
+ if err != nil || status != wire.StatusData || !bytes.Equal(body, probePattern(256)) {
+ t.Fatalf("clear B status=%d len=%d err=%v", status, len(body), err)
+ }
+ })
+ }
+}
diff --git a/core/cmd/dragontcp-server/chunk.go b/core/cmd/dragontcp-server/chunk.go
index a49486e..7b8883c 100644
--- a/core/cmd/dragontcp-server/chunk.go
+++ b/core/cmd/dragontcp-server/chunk.go
@@ -9,6 +9,7 @@ import (
"sync"
"time"
+ "dragontcp/internal/protocol"
"dragontcp/internal/wire"
)
@@ -55,11 +56,13 @@ func (s *streamSession) signalLocked() {
func (s *streamSession) touchLocked() { s.lastSeen = time.Now() }
func (s *streamSession) readTarget() {
- tmp := make([]byte, 64*1024)
+ ptr := protocol.BufferPool.Get().(*[]byte)
+ tmp := *ptr
+ defer protocol.BufferPool.Put(ptr)
for {
n, err := s.target.Read(tmp)
if n > 0 {
- data := append([]byte(nil), tmp[:n]...)
+ data := tmp[:n]
for len(data) > 0 {
s.mu.Lock()
for !s.closed && len(s.buf) >= s.maxBuffer {
diff --git a/core/cmd/dragontcp-server/chunk_test.go b/core/cmd/dragontcp-server/chunk_test.go
index 1801b33..1eeed28 100644
--- a/core/cmd/dragontcp-server/chunk_test.go
+++ b/core/cmd/dragontcp-server/chunk_test.go
@@ -1,8 +1,17 @@
package main
import (
+ "bytes"
"encoding/binary"
+ "fmt"
+ "io"
+ "net"
"testing"
+ "time"
+
+ "dragontcp/internal/cover"
+ "dragontcp/internal/protocol"
+ "dragontcp/internal/wire"
)
func TestParseOpenAllowsEmptyToken(t *testing.T) {
@@ -20,3 +29,175 @@ func TestParseOpenAllowsEmptyToken(t *testing.T) {
t.Fatalf("got token=%q host=%q port=%d", token, gotHost, port)
}
}
+
+func TestBinaryProfileProbeEndToEnd(t *testing.T) {
+ for n := 0; n < 256; n += 8 {
+ mask := byte(n)
+ server, client := net.Pipe()
+ clientResult := make(chan error, 1)
+ go func() {
+ defer client.Close()
+ var sid wire.SessionID
+ payload := make([]byte, 11)
+ copy(payload[:4], wire.ProbeMagic[:])
+ payload[4] = wire.ProbeKeepalive
+ if err := wire.WriteRequestProfile(client, wire.ModeProbe, sid, 1, payload, mask); err != nil {
+ clientResult <- err
+ return
+ }
+ status, _, err := wire.ReadResponseProfile(client, mask)
+ if err == nil && status != wire.StatusOK {
+ err = fmt.Errorf("status=%d", status)
+ }
+ clientResult <- err
+ }()
+
+ profiled, isXOR, gotMask, err := sniffWire(server)
+ if err != nil || isXOR || gotMask != mask {
+ t.Fatalf("mask %02x sniff: xor=%t gotMask=%02x err=%v", mask, isXOR, gotMask, err)
+ }
+ req, err := wire.ReadRequestProfile(profiled, gotMask)
+ if err == nil {
+ err = processWireRequest(profiled, req, "", false, nil, 0, nil, 1024, 0, 0, nil)
+ }
+ if err != nil {
+ t.Fatalf("mask %02x server: %v", mask, err)
+ }
+ if err := <-clientResult; err != nil {
+ t.Fatalf("mask %02x client: %v", mask, err)
+ }
+ _ = server.Close()
+ }
+}
+
+func TestXORProfileProbeEndToEnd(t *testing.T) {
+ for n := 0; n < 256; n++ {
+ mask := byte(n)
+ if ('U'^mask)&7 < 5 {
+ continue
+ }
+ server, client := net.Pipe()
+ clientResult := make(chan error, 1)
+ go func() {
+ defer client.Close()
+ if err := protocol.WriteRequestFrameProfile(client, 7, []byte("CPROBE -"), mask); err != nil {
+ clientResult <- err
+ return
+ }
+ id, payload, err := protocol.ReadResponseFrameProfile(client, mask)
+ if err == nil && (id != 7 || string(payload) != "PROBEOK") {
+ err = fmt.Errorf("id=%d payload=%q", id, payload)
+ }
+ clientResult <- err
+ }()
+
+ profiled, isXOR, gotMask, err := sniffWire(server)
+ if err != nil || !isXOR || gotMask != mask {
+ t.Fatalf("mask %02x sniff: xor=%t gotMask=%02x err=%v", mask, isXOR, gotMask, err)
+ }
+ handleXOR(profiled, gotMask, "", false, nil, 0, nil, 1024, 8, time.Millisecond, nil)
+ if err := <-clientResult; err != nil {
+ t.Fatalf("mask %02x client: %v", mask, err)
+ }
+ _ = server.Close()
+ }
+}
+
+func TestCoveredProfilesProbeEndToEnd(t *testing.T) {
+ for _, padding := range []uint16{0, 64, cover.MaxPadding} {
+ for _, xor := range []bool{false, true} {
+ profile := cover.Profile{Enabled: true, ID: 0x91e7, Padding: padding, HeaderMask: 0x6b, XOR: xor}
+ server, client := net.Pipe()
+ clientResult := make(chan error, 1)
+ go func() {
+ defer client.Close()
+ if err := cover.WritePreface(client, profile); err != nil {
+ clientResult <- err
+ return
+ }
+ if xor {
+ if err := protocol.WriteRequestFrameProfile(client, 11, []byte("CPROBE -"), profile.HeaderMask); err != nil {
+ clientResult <- err
+ return
+ }
+ id, payload, err := protocol.ReadResponseFrameProfile(client, profile.HeaderMask)
+ if err == nil && (id != 11 || string(payload) != "PROBEOK") {
+ err = fmt.Errorf("id=%d payload=%q", id, payload)
+ }
+ clientResult <- err
+ return
+ }
+
+ var sid wire.SessionID
+ payload := make([]byte, 11)
+ copy(payload[:4], wire.ProbeMagic[:])
+ payload[4] = wire.ProbeKeepalive
+ if err := wire.WriteRequestProfile(client, wire.ModeProbe, sid, 3, payload, profile.HeaderMask); err != nil {
+ clientResult <- err
+ return
+ }
+ status, _, err := wire.ReadResponseProfile(client, profile.HeaderMask)
+ if err == nil && status != wire.StatusOK {
+ err = fmt.Errorf("status=%d", status)
+ }
+ clientResult <- err
+ }()
+
+ profiled, gotXOR, gotMask, err := sniffWire(server)
+ if err != nil || gotXOR != xor || gotMask != profile.HeaderMask {
+ t.Fatalf("padding=%d xor=%t sniff got xor=%t mask=%02x err=%v", padding, xor, gotXOR, gotMask, err)
+ }
+ if xor {
+ handleXOR(profiled, gotMask, "", false, nil, 0, nil, 1024, 8, time.Millisecond, nil)
+ } else {
+ req, readErr := wire.ReadRequestProfile(profiled, gotMask)
+ if readErr == nil {
+ readErr = processWireRequest(profiled, req, "", false, nil, 0, nil, 1024, 0, 0, nil)
+ }
+ if readErr != nil {
+ t.Fatalf("padding=%d binary server: %v", padding, readErr)
+ }
+ }
+ if err := <-clientResult; err != nil {
+ t.Fatalf("padding=%d xor=%t client: %v", padding, xor, err)
+ }
+ _ = server.Close()
+ }
+ }
+}
+
+func TestSniffWireRecognizesAllHeaderProfiles(t *testing.T) {
+ test := func(firstTwo []byte, wantXOR bool, wantMask byte) {
+ server, client := net.Pipe()
+ defer server.Close()
+ go func() {
+ initial := make([]byte, 12)
+ copy(initial, firstTwo)
+ _, _ = client.Write(initial)
+ _ = client.Close()
+ }()
+
+ profiled, gotXOR, gotMask, err := sniffWire(server)
+ if err != nil {
+ t.Fatalf("header=%x: %v", firstTwo, err)
+ }
+ if gotXOR != wantXOR || gotMask != wantMask {
+ t.Fatalf("header=%x got xor=%t mask=%02x, want xor=%t mask=%02x", firstTwo, gotXOR, gotMask, wantXOR, wantMask)
+ }
+ replayed := make([]byte, 2)
+ if _, err := io.ReadFull(profiled, replayed); err != nil || !bytes.Equal(replayed, firstTwo) {
+ t.Fatalf("header=%x replay=%x err=%v", firstTwo, replayed, err)
+ }
+ }
+
+ for n := 0; n < 256; n += 8 {
+ mask := byte(n)
+ test([]byte{mask, 0xa7}, false, mask)
+ }
+ for n := 0; n < 256; n++ {
+ mask := byte(n)
+ if ('U'^mask)&7 >= 5 {
+ test([]byte{'U' ^ mask, 'P' ^ mask}, true, mask)
+ }
+ }
+}
diff --git a/core/cmd/dragontcp-server/main.go b/core/cmd/dragontcp-server/main.go
index ee2a083..c9fa3a1 100644
--- a/core/cmd/dragontcp-server/main.go
+++ b/core/cmd/dragontcp-server/main.go
@@ -15,11 +15,32 @@ import (
"time"
"dragontcp/internal/protocol"
- "dragontcp/internal/wire"
)
var active int64
+// idleDeadline avoids a SetDeadline system call for every small protocol
+// record. It refreshes halfway through the idle window, preserving idle-client
+// cleanup while making persistent high-throughput lanes substantially cheaper.
+type idleDeadline struct {
+ conn net.Conn
+ timeout time.Duration
+ next time.Time
+}
+
+func newIdleDeadline(conn net.Conn, timeout time.Duration) *idleDeadline {
+ return &idleDeadline{conn: conn, timeout: timeout}
+}
+
+func (d *idleDeadline) refresh() error {
+ now := time.Now()
+ if !d.next.IsZero() && now.Before(d.next.Add(-d.timeout/2)) {
+ return nil
+ }
+ d.next = now.Add(d.timeout)
+ return d.conn.SetDeadline(d.next)
+}
+
type dnsEntry struct {
ips []netip.Addr
expires time.Time
@@ -160,10 +181,11 @@ func handle(
tcpBuffer int,
slots chan struct{},
manager *streamManager,
+ bhttpManager *bhttpSessionManager,
xorManager *chunkManager,
chunkMax int,
bufferBytes int,
- chunkBuffered int,
+ xorBufferBytes int,
chunkPollWait time.Duration,
debug *serverDebug,
) {
@@ -176,46 +198,71 @@ func handle(
protocol.TuneTCP(conn)
protocol.TuneTCPBuffer(conn, tcpBuffer)
- // One listener serves both wires. The legacy XOR framing starts every
- // request with the ASCII magic "UP"; the binary framing starts with a mode
- // byte of 0-4, so the two are never ambiguous.
+ // One listener serves both wires and every startup-selected header profile.
+ // sniffWire partitions the full first-byte space so B and X remain
+ // unambiguous even when their legacy mode/UP bytes are masked.
_ = conn.SetDeadline(time.Now().Add(30 * time.Second))
- conn, isXOR, err := sniffWire(conn)
+ conn, isXOR, headerMask, err := sniffWire(conn)
if err != nil {
return
}
if isXOR {
if debug != nil && debug.enabled {
- debug.logf("WIRE peer=%v mode=xor", conn.RemoteAddr())
+ debug.logf("WIRE peer=%v mode=xor header_mask=%02x", conn.RemoteAddr(), headerMask)
}
- handleXOR(conn, token, allowPrivate, cache, tcpBuffer, xorManager,
- chunkMax, chunkBuffered, chunkPollWait, debug)
+ handleXOR(conn, headerMask, token, allowPrivate, cache, tcpBuffer, xorManager,
+ chunkMax, xorBufferBytes, chunkPollWait, debug)
return
}
+ clearPayload := false
+ if profiled, ok := conn.(interface{ ClearPayload() bool }); ok {
+ clearPayload = profiled.ClearPayload()
+ }
if debug != nil && debug.enabled {
- debug.logf("WIRE peer=%v mode=binary", conn.RemoteAddr())
+ debug.logf("WIRE peer=%v mode=binary header_mask=%02x clear_payload=%t", conn.RemoteAddr(), headerMask, clearPayload)
}
+ handleBinary(conn, headerMask, clearPayload, token, allowPrivate, cache, tcpBuffer, manager,
+ bhttpManager, chunkMax, bufferBytes, chunkPollWait, debug)
+}
+
+func acceptLoop(
+ ln net.Listener,
+ token string,
+ allowPrivate bool,
+ cache *dnsCache,
+ tcpBuffer int,
+ slots chan struct{},
+ manager *streamManager,
+ bhttpManager *bhttpSessionManager,
+ xorManager *chunkManager,
+ chunkMax int,
+ bufferBytes int,
+ xorBufferBytes int,
+ chunkPollWait time.Duration,
+ debug *serverDebug,
+) {
for {
- _ = conn.SetDeadline(time.Now().Add(30 * time.Second))
- req, err := wire.ReadRequest(conn)
+ conn, err := ln.Accept()
if err != nil {
- return
+ fmt.Fprintln(os.Stderr, "accept:", err)
+ continue
}
- if err := processWireRequest(
- conn,
- req,
- token,
- allowPrivate,
- cache,
- tcpBuffer,
- manager,
- chunkMax,
- bufferBytes,
- chunkPollWait,
- debug,
- ); err != nil {
- return
+
+ select {
+ case slots <- struct{}{}:
+ atomic.AddInt64(&active, 1)
+ if debug.enabled {
+ debug.logf("ACCEPT local=%v peer=%v active_connections=%d", conn.LocalAddr(), conn.RemoteAddr(), atomic.LoadInt64(&active))
+ }
+ go handle(conn, token, allowPrivate, cache, tcpBuffer, slots, manager,
+ bhttpManager, xorManager, chunkMax, bufferBytes, xorBufferBytes,
+ chunkPollWait, debug)
+ default:
+ if debug.enabled {
+ debug.errorf("REJECT peer=%v reason=max-connections", conn.RemoteAddr())
+ }
+ _ = conn.Close()
}
}
}
@@ -224,6 +271,7 @@ func main() {
var (
host = flag.String("host", "0.0.0.0", "listen host")
port = flag.Int("port", 53, "listen port")
+ portAlt = flag.Int("port-alt", 80, "second simultaneous listen port; 0 disables")
token = flag.String("token", "", "optional shared token")
maxConnections = flag.Int("max-connections", 20000, "max simultaneous tunnels")
allowPrivate = flag.Bool("allow-private", false, "allow private/loopback targets")
@@ -231,7 +279,7 @@ func main() {
dnsCacheSize = flag.Int("dns-cache-size", 4096, "maximum cached DNS hostnames")
tcpBuffer = flag.Int("tcp-buffer", 0, "optional TCP read/write buffer bytes; 0 keeps OS autotuning")
chunkMax = flag.Int("chunk-max", 1048576, "maximum adaptive chunk payload bytes (32 bytes to 1 MiB)")
- chunkBuffered = flag.Int("chunk-buffered", 256, "compatibility buffer units; 256 = about 16 MiB per active session")
+ chunkBuffered = flag.Int("chunk-buffered", 32, "per-session download buffer in 64 KiB units; 32 = about 2 MiB")
chunkPollWait = flag.Duration("chunk-poll-wait", 200*time.Millisecond, "server long-poll wait for chunk data")
sessionTimeout = flag.Duration("chunk-session-timeout", 2*time.Minute, "idle chunk session timeout")
debugEnabled = flag.Bool("debug", false, "log session/connect/errors and periodic statistics")
@@ -256,8 +304,23 @@ func main() {
os.Exit(1)
}
defer ln.Close()
-
+ listeners := []net.Listener{ln}
fmt.Printf("DragonTCP Go server listening on %s\n", listenAddr)
+ if *portAlt < 0 || *portAlt > 65535 {
+ fmt.Fprintln(os.Stderr, "--port-alt must be between 0 and 65535")
+ os.Exit(2)
+ }
+ if *portAlt != 0 && *portAlt != *port {
+ altAddr := net.JoinHostPort(*host, strconv.Itoa(*portAlt))
+ alt, altErr := net.Listen("tcp", altAddr)
+ if altErr != nil {
+ fmt.Fprintf(os.Stderr, "warning: secondary listener %s unavailable: %v\n", altAddr, altErr)
+ } else {
+ defer alt.Close()
+ listeners = append(listeners, alt)
+ fmt.Printf("DragonTCP Go server listening on %s\n", altAddr)
+ }
+ }
fmt.Printf("max_connections=%d tcp_buffer=%d\n", *maxConnections, *tcpBuffer)
slots := make(chan struct{}, *maxConnections)
@@ -271,45 +334,17 @@ func main() {
bufferBytes = 64 * 1024 * 1024
}
manager := newStreamManager(*sessionTimeout, debug)
+ bhttpManager := newBHTTPSessionManager(*sessionTimeout, *maxConnections)
xorManager := newChunkManager(*sessionTimeout, debug)
- fmt.Printf("binary_transport=true chunk_max=%d buffer_bytes=%d poll_wait=%s\n", *chunkMax, bufferBytes, chunkPollWait.String())
+ fmt.Printf("binary_transport=true bp_compat=true chunk_max=%d buffer_bytes=%d poll_wait=%s\n", *chunkMax, bufferBytes, chunkPollWait.String())
if debug.enabled {
fmt.Printf("debug=true debug_chunks=%t stats_interval=%s\n", debug.chunks, debug.statsEvery)
}
- for {
- conn, err := ln.Accept()
- if err != nil {
- fmt.Fprintln(os.Stderr, "accept:", err)
- continue
- }
-
- select {
- case slots <- struct{}{}:
- atomic.AddInt64(&active, 1)
- if debug.enabled {
- debug.logf("ACCEPT peer=%v active_connections=%d", conn.RemoteAddr(), atomic.LoadInt64(&active))
- }
- go handle(
- conn,
- *token,
- *allowPrivate,
- cache,
- *tcpBuffer,
- slots,
- manager,
- xorManager,
- *chunkMax,
- bufferBytes,
- *chunkBuffered,
- *chunkPollWait,
- debug,
- )
- default:
- if debug.enabled {
- debug.errorf("REJECT peer=%v reason=max-connections", conn.RemoteAddr())
- }
- _ = conn.Close()
- }
+ for _, listener := range listeners {
+ go acceptLoop(listener, *token, *allowPrivate, cache, *tcpBuffer, slots,
+ manager, bhttpManager, xorManager, *chunkMax, bufferBytes,
+ bufferBytes, *chunkPollWait, debug)
}
+ select {}
}
diff --git a/core/cmd/dragontcp-server/xorchunk.go b/core/cmd/dragontcp-server/xorchunk.go
index 63a45cb..7e9337a 100644
--- a/core/cmd/dragontcp-server/xorchunk.go
+++ b/core/cmd/dragontcp-server/xorchunk.go
@@ -17,6 +17,7 @@ import (
"sync"
"time"
+ "dragontcp/internal/cover"
"dragontcp/internal/protocol"
)
@@ -24,11 +25,12 @@ type chunkSession struct {
id string
target net.Conn
maxChunk int
- maxChunks int
+ maxBuffer int
mu sync.Mutex
notify chan struct{}
chunks map[uint64][]byte
+ buffered int
nextDown uint64
eof bool
closed bool
@@ -42,14 +44,28 @@ type chunkSession struct {
haveLastUp bool
}
-func newChunkSession(id string, target net.Conn, maxChunk, maxChunks int, debug *serverDebug) *chunkSession {
+func newChunkSession(id string, target net.Conn, maxChunk, maxBuffer int, debug *serverDebug) *chunkSession {
+ if maxBuffer < maxChunk {
+ maxBuffer = maxChunk
+ }
+ readSize := maxChunk
+ if readSize > 64*1024 {
+ readSize = 64 * 1024
+ }
+ mapCapacity := maxBuffer / readSize
+ if mapCapacity < 1 {
+ mapCapacity = 1
+ }
+ if mapCapacity > 256 {
+ mapCapacity = 256
+ }
s := &chunkSession{
id: id,
target: target,
maxChunk: maxChunk,
- maxChunks: maxChunks,
+ maxBuffer: maxBuffer,
notify: make(chan struct{}),
- chunks: make(map[uint64][]byte, maxChunks),
+ chunks: make(map[uint64][]byte, mapCapacity),
lastSeen: time.Now(),
debug: debug,
}
@@ -73,7 +89,12 @@ func (s *chunkSession) touch() {
}
func (s *chunkSession) readTarget() {
- buf := make([]byte, s.maxChunk)
+ ptr := protocol.BufferPool.Get().(*[]byte)
+ buf := *ptr
+ defer protocol.BufferPool.Put(ptr)
+ if s.maxChunk < len(buf) {
+ buf = buf[:s.maxChunk]
+ }
for {
n, err := s.target.Read(buf)
@@ -89,10 +110,11 @@ func (s *chunkSession) readTarget() {
s.mu.Unlock()
return
}
- if len(s.chunks) < s.maxChunks {
+ if s.buffered+len(data) <= s.maxBuffer {
seq := s.nextDown
s.nextDown++
s.chunks[seq] = data
+ s.buffered += len(data)
s.touchLocked()
s.signalLocked()
s.mu.Unlock()
@@ -181,6 +203,7 @@ func (s *chunkSession) pull(want uint64, ack int64, offset, limit int, wait time
removed := false
for seq := range s.chunks {
if seq <= uint64(ack) {
+ s.buffered -= len(s.chunks[seq])
delete(s.chunks, seq)
removed = true
}
@@ -333,7 +356,8 @@ func decodeWireToken(token string) string {
}
func isChunkCommand(payload []byte) bool {
- return bytes.HasPrefix(payload, []byte("COPEN ")) ||
+ return bytes.HasPrefix(payload, []byte("CPROBE ")) ||
+ bytes.HasPrefix(payload, []byte("COPEN ")) ||
bytes.HasPrefix(payload, []byte("CPUSH ")) ||
bytes.HasPrefix(payload, []byte("CPULL ")) ||
bytes.HasPrefix(payload, []byte("CCLOSE "))
@@ -349,10 +373,21 @@ func processChunkCommand(
tcpBuffer int,
manager *chunkManager,
maxChunk int,
- maxBufferedChunks int,
+ maxBufferedBytes int,
pollWait time.Duration,
debug *serverDebug,
) error {
+ if bytes.HasPrefix(payload, []byte("CPROBE ")) {
+ parts := strings.Fields(string(payload))
+ if len(parts) != 2 {
+ return protocol.WriteResponseFrame(conn, requestID, []byte("ERR bad CPROBE"))
+ }
+ if !tokenEqual(decodeWireToken(parts[1]), token) {
+ return protocol.WriteResponseFrame(conn, requestID, []byte("ERR authentication failed"))
+ }
+ return protocol.WriteResponseFrame(conn, requestID, []byte("PROBEOK"))
+ }
+
if bytes.HasPrefix(payload, []byte("COPEN ")) {
parts := strings.Fields(string(payload))
if len(parts) != 5 {
@@ -378,7 +413,7 @@ func processChunkCommand(
return protocol.WriteResponseFrame(conn, requestID, []byte("ERR "+err.Error()))
}
- session := newChunkSession(sid, target, maxChunk, maxBufferedChunks, debug)
+ session := newChunkSession(sid, target, maxChunk, maxBufferedBytes, debug)
if err := manager.add(sid, session); err != nil {
session.close()
if debug != nil && debug.enabled {
@@ -521,40 +556,78 @@ func processChunkCommand(
// beyond the magic would be lost.
type prefixedConn struct {
net.Conn
- r io.Reader
+ r io.Reader
+ headerMask byte
+ cover cover.Profile
}
-func (p *prefixedConn) Read(b []byte) (int, error) { return p.r.Read(b) }
+func (p *prefixedConn) Read(b []byte) (int, error) { return p.r.Read(b) }
+func (p *prefixedConn) HeaderMask() byte { return p.headerMask }
+func (p *prefixedConn) CoverProfile() cover.Profile { return p.cover }
+func (p *prefixedConn) ClearPayload() bool { return p.cover.Clear }
-// sniffWire reads the two magic bytes and reports whether this connection
-// speaks the legacy XOR framing. The returned conn replays them.
-func sniffWire(conn net.Conn) (net.Conn, bool, error) {
- var magic [2]byte
- if _, err := io.ReadFull(conn, magic[:]); err != nil {
- return conn, false, err
+// sniffWire first checks for the optional self-describing cover preface. If it
+// is absent, the bytes are replayed and the legacy/direct B/X classifier is
+// used unchanged.
+func sniffWire(conn net.Conn) (net.Conn, bool, byte, error) {
+ var initial [cover.PrefaceSize]byte
+ if _, err := io.ReadFull(conn, initial[:]); err != nil {
+ return conn, false, 0, err
}
- replayed := &prefixedConn{Conn: conn, r: io.MultiReader(bytes.NewReader(magic[:]), conn)}
- return replayed, magic[0] == 'U' && magic[1] == 'P', nil
+ if profile, ok := cover.DecodePreface(initial); ok {
+ if profile.Padding > 0 {
+ padding := make([]byte, int(profile.Padding))
+ if _, err := io.ReadFull(conn, padding); err != nil {
+ return conn, false, 0, err
+ }
+ }
+ profiled := &prefixedConn{Conn: conn, r: conn, headerMask: profile.HeaderMask, cover: profile}
+ return profiled, profile.XOR, profile.HeaderMask, nil
+ }
+
+ magic := initial[:2]
+ replay := io.MultiReader(bytes.NewReader(initial[:]), conn)
+
+ if magic[0]&7 >= 5 {
+ mask := magic[0] ^ 'U'
+ if magic[1]^mask != 'P' {
+ return conn, false, 0, fmt.Errorf("unknown wire header")
+ }
+ replayed := &prefixedConn{Conn: conn, r: replay, headerMask: mask}
+ return replayed, true, mask, nil
+ }
+
+ mask := magic[0] & 0xf8
+ mode := magic[0] ^ mask
+ if mode > 4 {
+ return conn, false, 0, fmt.Errorf("unknown binary mode")
+ }
+ replayed := &prefixedConn{Conn: conn, r: replay, headerMask: mask}
+ return replayed, false, mask, nil
}
// handleXOR serves one connection speaking UP/OK + XOR 0xAD: the v4 chunk
// commands, plus the TUNNEL/TUNNEL2 stream commands.
func handleXOR(
conn net.Conn,
+ headerMask byte,
token string,
allowPrivate bool,
cache *dnsCache,
tcpBuffer int,
manager *chunkManager,
chunkMax int,
- chunkBuffered int,
+ bufferBytes int,
chunkPollWait time.Duration,
debug *serverDebug,
) {
+ deadline := newIdleDeadline(conn, 20*time.Second)
for {
- _ = conn.SetDeadline(time.Now().Add(20 * time.Second))
+ if deadline.refresh() != nil {
+ return
+ }
- requestID, _, payload, err := protocol.ReadRequestFrame(conn)
+ requestID, _, payload, err := protocol.ReadRequestFrameProfile(conn, headerMask)
if err != nil {
if debug != nil && debug.enabled && err != io.EOF {
debug.errorf("peer=%v read XOR request: %v", conn.RemoteAddr(), err)
@@ -565,7 +638,7 @@ func handleXOR(
if isChunkCommand(payload) {
if err := processChunkCommand(
conn, requestID, payload, token, allowPrivate, cache, tcpBuffer,
- manager, chunkMax, chunkBuffered, chunkPollWait, debug,
+ manager, chunkMax, bufferBytes, chunkPollWait, debug,
); err != nil {
return
}
diff --git a/core/internal/cover/profile.go b/core/internal/cover/profile.go
new file mode 100644
index 0000000..dae68f2
--- /dev/null
+++ b/core/internal/cover/profile.go
@@ -0,0 +1,141 @@
+// Package cover implements the optional connection preface used by startup
+// profile discovery. Legacy connections have no preface and remain supported.
+package cover
+
+import (
+ "crypto/rand"
+ "crypto/sha256"
+ "encoding/binary"
+ "fmt"
+ "io"
+)
+
+const (
+ PrefaceSize = 12
+ MaxPadding = 4096
+)
+
+// Profile is selected once during startup and then reused unchanged. Padding
+// bytes are freshly random on each physical connection, but their length and
+// all header fields remain fixed.
+type Profile struct {
+ Enabled bool
+ ID uint16
+ Padding uint16
+ HeaderMask byte
+ XOR bool
+ Clear bool
+}
+
+func (p Profile) String() string {
+ if !p.Enabled {
+ return "direct"
+ }
+ encoding := "masked"
+ if p.Clear {
+ encoding = "clear"
+ }
+ return fmt.Sprintf("cover-%04x/pad-%d/%s", p.ID, p.Padding, encoding)
+}
+
+func key(id uint16) [32]byte {
+ var seed [16]byte
+ copy(seed[:12], []byte("DragonTCP-C3"))
+ binary.BigEndian.PutUint16(seed[12:14], id)
+ seed[14], seed[15] = byte(id)^0x6d, byte(id>>8)^0xb2
+ return sha256.Sum256(seed[:])
+}
+
+// EncodePreface returns the fixed-size, self-describing portion. The first two
+// bytes are the mutable profile ID; all metadata after them is masked.
+func EncodePreface(p Profile) ([PrefaceSize]byte, error) {
+ var out [PrefaceSize]byte
+ if !p.Enabled {
+ return out, fmt.Errorf("cover profile is disabled")
+ }
+ if p.Padding > MaxPadding {
+ return out, fmt.Errorf("cover padding too large: %d", p.Padding)
+ }
+
+ binary.BigEndian.PutUint16(out[0:2], p.ID)
+ var plain [10]byte
+ copy(plain[0:4], []byte("DTC3"))
+ if p.XOR {
+ plain[4] |= 1
+ }
+ if p.Clear {
+ plain[4] |= 2
+ }
+ plain[5] = p.HeaderMask
+ binary.BigEndian.PutUint16(plain[6:8], p.Padding)
+ plain[8] = plain[4] ^ plain[5] ^ 0xa5
+ plain[9] = plain[6] ^ plain[7] ^ 0x5a
+ k := key(p.ID)
+ for i := range plain {
+ out[2+i] = plain[i] ^ k[i]
+ }
+ return out, nil
+}
+
+// DecodePreface recognizes an encoded cover profile. ok=false means the bytes
+// belong to a legacy/direct connection and must be replayed unchanged.
+func DecodePreface(in [PrefaceSize]byte) (p Profile, ok bool) {
+ id := binary.BigEndian.Uint16(in[0:2])
+ k := key(id)
+ var plain [10]byte
+ for i := range plain {
+ plain[i] = in[2+i] ^ k[i]
+ }
+ if string(plain[0:4]) != "DTC3" || plain[4]&^byte(3) != 0 {
+ return Profile{}, false
+ }
+ if plain[8] != plain[4]^plain[5]^0xa5 || plain[9] != plain[6]^plain[7]^0x5a {
+ return Profile{}, false
+ }
+ padding := binary.BigEndian.Uint16(plain[6:8])
+ if padding > MaxPadding {
+ return Profile{}, false
+ }
+ return Profile{
+ Enabled: true,
+ ID: id,
+ Padding: padding,
+ HeaderMask: plain[5],
+ XOR: plain[4]&1 != 0,
+ Clear: plain[4]&2 != 0,
+ }, true
+}
+
+// WritePreface sends the encoded profile followed by its fixed amount of
+// random padding.
+func WritePreface(w io.Writer, p Profile) error {
+ if !p.Enabled {
+ return nil
+ }
+ preface, err := EncodePreface(p)
+ if err != nil {
+ return err
+ }
+ packet := make([]byte, PrefaceSize+int(p.Padding))
+ copy(packet, preface[:])
+ if p.Padding > 0 {
+ if _, err := rand.Read(packet[PrefaceSize:]); err != nil {
+ return err
+ }
+ }
+ return writeAll(w, packet)
+}
+
+func writeAll(w io.Writer, b []byte) error {
+ for len(b) > 0 {
+ n, err := w.Write(b)
+ if err != nil {
+ return err
+ }
+ if n <= 0 {
+ return io.ErrShortWrite
+ }
+ b = b[n:]
+ }
+ return nil
+}
diff --git a/core/internal/cover/profile_test.go b/core/internal/cover/profile_test.go
new file mode 100644
index 0000000..29d4595
--- /dev/null
+++ b/core/internal/cover/profile_test.go
@@ -0,0 +1,41 @@
+package cover
+
+import (
+ "bytes"
+ "testing"
+)
+
+func TestProfileRoundTripAcrossRange(t *testing.T) {
+ for id := 0; id < 65536; id += 257 {
+ for _, xor := range []bool{false, true} {
+ for _, clear := range []bool{false, true} {
+ want := Profile{Enabled: true, ID: uint16(id), Padding: uint16(id % (MaxPadding + 1)), HeaderMask: byte(id), XOR: xor, Clear: clear}
+ encoded, err := EncodePreface(want)
+ if err != nil {
+ t.Fatal(err)
+ }
+ got, ok := DecodePreface(encoded)
+ if !ok || got != want {
+ t.Fatalf("id=%04x xor=%t clear=%t got=%+v ok=%t", id, xor, clear, got, ok)
+ }
+ }
+ }
+ }
+}
+
+func TestWritePrefaceIncludesFixedPadding(t *testing.T) {
+ p := Profile{Enabled: true, ID: 0x1234, Padding: 64, HeaderMask: 0x9a, XOR: true}
+ var out bytes.Buffer
+ if err := WritePreface(&out, p); err != nil {
+ t.Fatal(err)
+ }
+ if out.Len() != PrefaceSize+64 {
+ t.Fatalf("length=%d", out.Len())
+ }
+ var encoded [PrefaceSize]byte
+ copy(encoded[:], out.Bytes())
+ got, ok := DecodePreface(encoded)
+ if !ok || got != p {
+ t.Fatalf("got=%+v ok=%t", got, ok)
+ }
+}
diff --git a/core/internal/protocol/protocol.go b/core/internal/protocol/protocol.go
index 555dcfd..18c9acb 100644
--- a/core/internal/protocol/protocol.go
+++ b/core/internal/protocol/protocol.go
@@ -31,12 +31,17 @@ var BufferPool = sync.Pool{
}
func ReadRequestFrame(r io.Reader) (uint32, uint32, []byte, error) {
+ return ReadRequestFrameProfile(r, 0)
+}
+
+// ReadRequestFrameProfile decodes the UP magic after applying headerMask.
+func ReadRequestFrameProfile(r io.Reader, headerMask byte) (uint32, uint32, []byte, error) {
var header [14]byte
if _, err := io.ReadFull(r, header[:]); err != nil {
return 0, 0, nil, err
}
- if header[0] != 'U' || header[1] != 'P' {
+ if header[0]^headerMask != 'U' || header[1]^headerMask != 'P' {
return 0, 0, nil, errors.New("bad request magic")
}
@@ -58,11 +63,16 @@ func ReadRequestFrame(r io.Reader) (uint32, uint32, []byte, error) {
}
func WriteRequestFrame(w io.Writer, requestID uint32, payload []byte) error {
+ return WriteRequestFrameProfile(w, requestID, payload, 0)
+}
+
+// WriteRequestFrameProfile masks the two-byte UP magic with headerMask.
+func WriteRequestFrameProfile(w io.Writer, requestID uint32, payload []byte, headerMask byte) error {
if len(payload) > MaxHandshake {
return errors.New("request frame payload too large")
}
packet := make([]byte, 14+len(payload))
- packet[0], packet[1] = 'U', 'P'
+ packet[0], packet[1] = 'U'^headerMask, 'P'^headerMask
binary.BigEndian.PutUint32(packet[2:6], requestID)
binary.BigEndian.PutUint32(packet[6:10], 0)
binary.BigEndian.PutUint32(packet[10:14], uint32(len(payload)))
@@ -72,12 +82,17 @@ func WriteRequestFrame(w io.Writer, requestID uint32, payload []byte) error {
}
func ReadResponseFrame(r io.Reader) (uint32, []byte, error) {
+ return ReadResponseFrameProfile(r, 0)
+}
+
+// ReadResponseFrameProfile decodes the OK magic after applying headerMask.
+func ReadResponseFrameProfile(r io.Reader, headerMask byte) (uint32, []byte, error) {
var header [10]byte
if _, err := io.ReadFull(r, header[:]); err != nil {
return 0, nil, err
}
- if header[0] != 'O' || header[1] != 'K' {
+ if header[0]^headerMask != 'O' || header[1]^headerMask != 'K' {
return 0, nil, fmt.Errorf("bad response magic: %q", header[:2])
}
@@ -98,11 +113,16 @@ func ReadResponseFrame(r io.Reader) (uint32, []byte, error) {
}
func WriteResponseFrame(w io.Writer, requestID uint32, payload []byte) error {
+ return WriteResponseFrameProfile(w, requestID, payload, writerHeaderMask(w))
+}
+
+// WriteResponseFrameProfile masks the two-byte OK magic with headerMask.
+func WriteResponseFrameProfile(w io.Writer, requestID uint32, payload []byte, headerMask byte) error {
if len(payload) > MaxHandshake {
return errors.New("response frame payload too large")
}
packet := make([]byte, 10+len(payload))
- packet[0], packet[1] = 'O', 'K'
+ packet[0], packet[1] = 'O'^headerMask, 'K'^headerMask
binary.BigEndian.PutUint32(packet[2:6], requestID)
binary.BigEndian.PutUint32(packet[6:10], uint32(len(payload)))
copy(packet[10:], payload)
@@ -121,6 +141,13 @@ func writeAll(w io.Writer, b []byte) error {
return nil
}
+func writerHeaderMask(w io.Writer) byte {
+ if profiled, ok := w.(interface{ HeaderMask() byte }); ok {
+ return profiled.HeaderMask()
+ }
+ return 0
+}
+
func CopyXOR(dst net.Conn, src net.Conn) error {
ptr := BufferPool.Get().(*[]byte)
buf := *ptr
diff --git a/core/internal/protocol/protocol_test.go b/core/internal/protocol/protocol_test.go
new file mode 100644
index 0000000..af6908b
--- /dev/null
+++ b/core/internal/protocol/protocol_test.go
@@ -0,0 +1,50 @@
+package protocol
+
+import (
+ "bytes"
+ "testing"
+)
+
+func TestXORHeaderProfilesRoundTrip(t *testing.T) {
+ for n := 0; n < 256; n++ {
+ mask := byte(n)
+ if ('U'^mask)&7 < 5 {
+ continue
+ }
+
+ var request bytes.Buffer
+ if err := WriteRequestFrameProfile(&request, 7, []byte("CPROBE -"), mask); err != nil {
+ t.Fatal(err)
+ }
+ requestID, _, payload, err := ReadRequestFrameProfile(&request, mask)
+ if err != nil || requestID != 7 || !bytes.Equal(payload, []byte("CPROBE -")) {
+ t.Fatalf("mask %02x request did not round-trip: id=%d payload=%q err=%v", mask, requestID, payload, err)
+ }
+
+ var response bytes.Buffer
+ if err := WriteResponseFrameProfile(&response, 7, []byte("PROBEOK"), mask); err != nil {
+ t.Fatal(err)
+ }
+ responseID, payload, err := ReadResponseFrameProfile(&response, mask)
+ if err != nil || responseID != 7 || !bytes.Equal(payload, []byte("PROBEOK")) {
+ t.Fatalf("mask %02x response did not round-trip: id=%d payload=%q err=%v", mask, responseID, payload, err)
+ }
+ }
+}
+
+type profiledBuffer struct {
+ bytes.Buffer
+ mask byte
+}
+
+func (b *profiledBuffer) HeaderMask() byte { return b.mask }
+
+func TestServerResponseUsesConnectionProfile(t *testing.T) {
+ profiled := &profiledBuffer{mask: 0x3a}
+ if err := WriteResponseFrame(profiled, 9, []byte("ok")); err != nil {
+ t.Fatal(err)
+ }
+ if got := profiled.Bytes()[0]; got != 'O'^profiled.mask {
+ t.Fatalf("first byte=%02x, want %02x", got, byte('O')^profiled.mask)
+ }
+}
diff --git a/core/internal/wire/protocol.go b/core/internal/wire/protocol.go
index 7e8730c..bf83572 100644
--- a/core/internal/wire/protocol.go
+++ b/core/internal/wire/protocol.go
@@ -6,6 +6,7 @@ import (
"errors"
"fmt"
"io"
+ "net"
)
const (
@@ -72,12 +73,36 @@ func MaskInPlace(data []byte, sid SessionID, mode byte, seq uint64, response boo
}
func WriteRequest(w io.Writer, mode byte, sid SessionID, seq uint64, plaintext []byte) error {
+ return WriteRequestProfile(w, mode, sid, seq, plaintext, 0)
+}
+
+// WriteRequestProfile writes a binary request whose first byte is XORed with
+// headerMask. The remaining framing and payload encoding stay unchanged.
+// Masks are selected once at client startup and then remain fixed.
+func WriteRequestProfile(w io.Writer, mode byte, sid SessionID, seq uint64, plaintext []byte, headerMask byte) error {
+ return WriteRequestProfileEncoding(w, mode, sid, seq, plaintext, headerMask, false)
+}
+
+// WriteRequestProfileEncoding optionally leaves the payload clear. Clear mode
+// is signalled by the connection cover preface, so legacy peers continue to use
+// the SHA-256 compatibility mask unchanged.
+func WriteRequestProfileEncoding(w io.Writer, mode byte, sid SessionID, seq uint64, plaintext []byte, headerMask byte, clear bool) error {
if len(plaintext) > MaxPayload {
return fmt.Errorf("request payload too large: %d", len(plaintext))
}
+ if clear {
+ var header [RequestHeaderSize]byte
+ header[0] = mode ^ headerMask
+ copy(header[1:17], sid[:])
+ binary.BigEndian.PutUint64(header[17:25], seq)
+ binary.BigEndian.PutUint32(header[25:29], uint32(len(plaintext)))
+ buffers := net.Buffers{header[:], plaintext}
+ _, err := buffers.WriteTo(w)
+ return err
+ }
packet := make([]byte, RequestHeaderSize+len(plaintext))
- packet[0] = mode
+ packet[0] = mode ^ headerMask
copy(packet[1:17], sid[:])
binary.BigEndian.PutUint64(packet[17:25], seq)
binary.BigEndian.PutUint32(packet[25:29], uint32(len(plaintext)))
@@ -87,13 +112,25 @@ func WriteRequest(w io.Writer, mode byte, sid SessionID, seq uint64, plaintext [
}
func ReadRequest(r io.Reader) (Request, error) {
+ return ReadRequestProfile(r, 0)
+}
+
+// ReadRequestProfile decodes a request written with WriteRequestProfile.
+func ReadRequestProfile(r io.Reader, headerMask byte) (Request, error) {
+ return ReadRequestProfileEncoding(r, headerMask, false)
+}
+
+func ReadRequestProfileEncoding(r io.Reader, headerMask byte, clear bool) (Request, error) {
var req Request
var header [RequestHeaderSize]byte
if _, err := io.ReadFull(r, header[:]); err != nil {
return req, err
}
- req.Mode = header[0]
+ req.Mode = header[0] ^ headerMask
+ if req.Mode > ModeClose {
+ return req, errors.New("unknown request mode")
+ }
copy(req.Session[:], header[1:17])
req.Seq = binary.BigEndian.Uint64(header[17:25])
n := binary.BigEndian.Uint32(header[25:29])
@@ -106,28 +143,53 @@ func ReadRequest(r io.Reader) (Request, error) {
if _, err := io.ReadFull(r, req.Payload); err != nil {
return req, err
}
- MaskInPlace(req.Payload, req.Session, req.Mode, req.Seq, false)
+ if !clear {
+ MaskInPlace(req.Payload, req.Session, req.Mode, req.Seq, false)
+ }
}
return req, nil
}
func WriteResponse(w io.Writer, status byte, body []byte) error {
+ return WriteResponseProfile(w, status, body, writerHeaderMask(w))
+}
+
+// WriteResponseProfile writes a response using the selected first-byte mask.
+func WriteResponseProfile(w io.Writer, status byte, body []byte, headerMask byte) error {
if len(body) > MaxPayload {
return fmt.Errorf("response body too large: %d", len(body))
}
packet := make([]byte, ResponseHeaderSize+len(body))
- packet[0] = status
+ packet[0] = status ^ headerMask
binary.BigEndian.PutUint32(packet[1:5], uint32(len(body)))
copy(packet[5:], body)
return writeAll(w, packet)
}
func WriteMaskedResponse(w io.Writer, status byte, body []byte, sid SessionID, mode byte, seq uint64) error {
+ return WriteMaskedResponseProfileEncoding(w, status, body, sid, mode, seq, writerHeaderMask(w), writerClearPayload(w))
+}
+
+// WriteMaskedResponseProfile combines the normal payload mask with the
+// selected first-byte header mask.
+func WriteMaskedResponseProfile(w io.Writer, status byte, body []byte, sid SessionID, mode byte, seq uint64, headerMask byte) error {
+ return WriteMaskedResponseProfileEncoding(w, status, body, sid, mode, seq, headerMask, false)
+}
+
+func WriteMaskedResponseProfileEncoding(w io.Writer, status byte, body []byte, sid SessionID, mode byte, seq uint64, headerMask byte, clear bool) error {
if len(body) > MaxPayload {
return fmt.Errorf("response body too large: %d", len(body))
}
+ if clear {
+ var header [ResponseHeaderSize]byte
+ header[0] = status ^ headerMask
+ binary.BigEndian.PutUint32(header[1:5], uint32(len(body)))
+ buffers := net.Buffers{header[:], body}
+ _, err := buffers.WriteTo(w)
+ return err
+ }
packet := make([]byte, ResponseHeaderSize+len(body))
- packet[0] = status
+ packet[0] = status ^ headerMask
binary.BigEndian.PutUint32(packet[1:5], uint32(len(body)))
copy(packet[5:], body)
MaskInPlace(packet[5:], sid, mode, seq, true)
@@ -135,6 +197,11 @@ func WriteMaskedResponse(w io.Writer, status byte, body []byte, sid SessionID, m
}
func ReadResponse(r io.Reader) (byte, []byte, error) {
+ return ReadResponseProfile(r, 0)
+}
+
+// ReadResponseProfile decodes a response written with a header profile.
+func ReadResponseProfile(r io.Reader, headerMask byte) (byte, []byte, error) {
var header [ResponseHeaderSize]byte
if _, err := io.ReadFull(r, header[:]); err != nil {
return 0, nil, err
@@ -150,16 +217,19 @@ func ReadResponse(r io.Reader) (byte, []byte, error) {
return 0, nil, err
}
}
- return header[0], body, nil
+ status := header[0] ^ headerMask
+ if status > StatusEOF {
+ return 0, nil, errors.New("unknown response status")
+ }
+ return status, body, nil
}
func DecodeMaskedResponse(status byte, body []byte, sid SessionID, mode byte, seq uint64) []byte {
if len(body) == 0 || status == StatusError {
return body
}
- out := append([]byte(nil), body...)
- MaskInPlace(out, sid, mode, seq, true)
- return out
+ MaskInPlace(body, sid, mode, seq, true)
+ return body
}
func writeAll(w io.Writer, b []byte) error {
@@ -175,3 +245,17 @@ func writeAll(w io.Writer, b []byte) error {
}
return nil
}
+
+func writerHeaderMask(w io.Writer) byte {
+ if profiled, ok := w.(interface{ HeaderMask() byte }); ok {
+ return profiled.HeaderMask()
+ }
+ return 0
+}
+
+func writerClearPayload(w io.Writer) bool {
+ if profiled, ok := w.(interface{ ClearPayload() bool }); ok {
+ return profiled.ClearPayload()
+ }
+ return false
+}
diff --git a/core/internal/wire/protocol_test.go b/core/internal/wire/protocol_test.go
index 8302467..c902baf 100644
--- a/core/internal/wire/protocol_test.go
+++ b/core/internal/wire/protocol_test.go
@@ -1,29 +1,105 @@
package wire
import (
- "bytes"
- "testing"
+ "bytes"
+ "io"
+ "testing"
)
func TestMaskChangesWithSequenceAndRoundTrips(t *testing.T) {
- var sid SessionID
- for i := range sid { sid[i] = byte(i+1) }
- plain := bytes.Repeat([]byte("DragonTCP"), 100)
- a := append([]byte(nil), plain...)
- b := append([]byte(nil), plain...)
- MaskInPlace(a, sid, ModeUpload, 1, false)
- MaskInPlace(b, sid, ModeUpload, 2, false)
- if bytes.Equal(a, b) { t.Fatal("different sequences produced identical wire bytes") }
- MaskInPlace(a, sid, ModeUpload, 1, false)
- if !bytes.Equal(a, plain) { t.Fatal("mask did not round-trip") }
+ var sid SessionID
+ for i := range sid {
+ sid[i] = byte(i + 1)
+ }
+ plain := bytes.Repeat([]byte("DragonTCP"), 100)
+ a := append([]byte(nil), plain...)
+ b := append([]byte(nil), plain...)
+ MaskInPlace(a, sid, ModeUpload, 1, false)
+ MaskInPlace(b, sid, ModeUpload, 2, false)
+ if bytes.Equal(a, b) {
+ t.Fatal("different sequences produced identical wire bytes")
+ }
+ MaskInPlace(a, sid, ModeUpload, 1, false)
+ if !bytes.Equal(a, plain) {
+ t.Fatal("mask did not round-trip")
+ }
+}
+
+func TestBinaryHeaderProfilesRoundTrip(t *testing.T) {
+ var sid SessionID
+ for i := range sid {
+ sid[i] = byte(i + 1)
+ }
+ for n := 0; n < 256; n += 8 {
+ mask := byte(n)
+ var request bytes.Buffer
+ if err := WriteRequestProfile(&request, ModeUpload, sid, 42, []byte("payload"), mask); err != nil {
+ t.Fatal(err)
+ }
+ if got := request.Bytes()[0]; got != ModeUpload^mask {
+ t.Fatalf("mask %02x first byte=%02x", mask, got)
+ }
+ req, err := ReadRequestProfile(&request, mask)
+ if err != nil {
+ t.Fatalf("mask %02x: %v", mask, err)
+ }
+ if req.Mode != ModeUpload || req.Seq != 42 || !bytes.Equal(req.Payload, []byte("payload")) {
+ t.Fatalf("mask %02x request did not round-trip", mask)
+ }
+
+ var response bytes.Buffer
+ if err := WriteResponseProfile(&response, StatusOK, []byte("ok"), mask); err != nil {
+ t.Fatal(err)
+ }
+ status, body, err := ReadResponseProfile(&response, mask)
+ if err != nil || status != StatusOK || !bytes.Equal(body, []byte("ok")) {
+ t.Fatalf("mask %02x response did not round-trip: status=%d body=%q err=%v", mask, status, body, err)
+ }
+ }
+}
+
+type profiledBuffer struct {
+ bytes.Buffer
+ mask byte
+}
+
+func (b *profiledBuffer) HeaderMask() byte { return b.mask }
+
+func TestServerResponseUsesConnectionProfile(t *testing.T) {
+ profiled := &profiledBuffer{mask: 0xa0}
+ if err := WriteResponse(profiled, StatusOK, []byte("ok")); err != nil {
+ t.Fatal(err)
+ }
+ if got := profiled.Bytes()[0]; got != StatusOK^profiled.mask {
+ t.Fatalf("first byte=%02x, want %02x", got, StatusOK^profiled.mask)
+ }
}
func BenchmarkMask1MiB(b *testing.B) {
- var sid SessionID
- data := make([]byte, 1024*1024)
- b.SetBytes(int64(len(data)))
- b.ResetTimer()
- for i:=0;i 2*time.Second {
+ timeout = 2 * time.Second
+ }
+ lane := newTxnLane(serverAddr, opts.tcpBuffer, 1, timeout, opts.headerMask, opts.coverProfile)
+ defer lane.Close()
+ resp, err := lane.Do([]byte("CPROBE " + wireToken(token)))
+ if err != nil {
+ return false
+ }
+ if string(resp) == "PROBEOK" {
+ return true
+ }
+ // Servers predating profile discovery do not know CPROBE, but receiving a
+ // correctly framed error still proves that the legacy mask-zero header
+ // survived. The subsequent end-to-end probe remains authoritative.
+ return opts.headerMask == 0 && strings.HasPrefix(string(resp), "ERR expected TUNNEL")
+}
+
type chunkResult struct {
seq uint64
data []byte
@@ -425,7 +474,7 @@ func Open(serverAddr, token, targetHost string, targetPort int, opts Options) (n
c.upSizer = newAdaptiveSizer("upload", opts)
c.downSizer = newAdaptiveSizer("download", opts)
- c.pushLane = newTxnLane(serverAddr, opts.tcpBuffer, opts.reconnectEvery, opts.txnTimeout)
+ c.pushLane = newTxnLane(serverAddr, opts.tcpBuffer, opts.reconnectEvery, opts.txnTimeout, opts.headerMask, opts.coverProfile)
openPayload := []byte(fmt.Sprintf(
"COPEN %s %s %s %d",
@@ -464,7 +513,7 @@ func Open(serverAddr, token, targetHost string, targetPort int, opts Options) (n
c.pullLanes = make([]*txnLane, opts.pollers)
for i := 0; i < opts.pollers; i++ {
- lane := newTxnLane(serverAddr, opts.tcpBuffer, opts.reconnectEvery, opts.txnTimeout)
+ lane := newTxnLane(serverAddr, opts.tcpBuffer, opts.reconnectEvery, opts.txnTimeout, opts.headerMask, opts.coverProfile)
c.pullLanes[i] = lane
c.workers.Add(1)
go c.pullWorker(lane)
@@ -762,7 +811,7 @@ func (c *chunkConn) Close() error {
c.once.Do(func() {
c.cancel()
- lane := newTxnLane(c.serverAddr, c.opts.tcpBuffer, 1, c.opts.txnTimeout)
+ lane := newTxnLane(c.serverAddr, c.opts.tcpBuffer, 1, c.opts.txnTimeout, c.opts.headerMask, c.opts.coverProfile)
_, _ = doControl(lane, []byte(fmt.Sprintf("CCLOSE %s %s", wireToken(c.token), c.sid)))
lane.Close()