diff --git a/README.md b/README.md index 9b77b17..ce8c825 100644 --- a/README.md +++ b/README.md @@ -1,122 +1,120 @@ # DragonTCP Hybrid -DragonTCP is a tunnelling proxy that carries ordinary TCP traffic inside a -compact binary record protocol, normally over TCP port 53. It is made of three -pieces that all live in this repository: +DragonTCP carries ordinary TCP traffic inside a compact binary record protocol, +normally over TCP port 53. It has three parts: -* a **Go server** that runs on a Linux VPS and relays streams to their real - destinations, -* a **Go client** that runs on the phone and exposes a local HTTP/HTTPS proxy, -* an **Android app** that captures all device traffic with `VpnService` and - feeds it into that local proxy. +* a **Go server** on a Linux host that relays streams to their real destinations, +* a **Go client** that exposes a local HTTP/HTTPS proxy, +* an **Android app** that captures all device traffic with `VpnService`, runs a + userspace TCP/IP stack, and feeds everything into that local proxy. -This document explains the whole system: the wire format, both Go programs, the -Android app, how to build everything (Windows and Linux), how to run it, and how -to tune it. +This document is a complete reference for how all of it works. --- ## Table of contents -1. [What it does and why](#1-what-it-does-and-why) -2. [Security model — read this](#2-security-model--read-this) +1. [Overview](#1-overview) +2. [Security model](#2-security-model) 3. [Repository layout](#3-repository-layout) 4. [The wire protocol](#4-the-wire-protocol) 5. [The Go client](#5-the-go-client) 6. [The Go server](#6-the-go-server) 7. [The Android app](#7-the-android-app) -8. [Building on Windows](#8-building-on-windows) -9. [Building on Linux and macOS](#9-building-on-linux-and-macos) -10. [Running the server](#10-running-the-server) -11. [Running the client](#11-running-the-client) -12. [Tuning guide](#12-tuning-guide) -13. [Troubleshooting](#13-troubleshooting) -14. [Version history](#14-version-history) -15. [Testing and validation](#15-testing-and-validation) -16. [Licensing](#16-licensing) +8. [Building](#8-building) +9. [Running](#9-running) +10. [Configuration reference](#10-configuration-reference) +11. [Tuning](#11-tuning) +12. [Troubleshooting](#12-troubleshooting) +13. [Testing](#13-testing) +14. [Licensing](#14-licensing) --- -## 1. What it does and why +## 1. Overview -### The data path +### 1.1 The data path ```text Android apps (any app, unmodified) - | - | IP packets - v - Android VpnService TUN interface (10.77.0.2/32, MTU 1400) - | - | userspace TCP/IP reassembly - v + │ + │ IP packets + ▼ + VpnService TUN interface 10.77.0.2/32, MTU 1400 + │ + │ userspace TCP/IP reassembly + ▼ TunnelEngine (Kotlin, in-process) - | - | HTTP CONNECT to 127.0.0.1:8080 - v + │ + │ HTTP CONNECT to 127.0.0.1:8080 + ▼ dragontcp-client (Go, child process on the phone) - | - | DragonTCP binary records over TCP/53 - v - dragontcp-server (Go, on the VPS) - | - | plain TCP - v - destination website + │ + │ DragonTCP binary records over TCP/53 + ▼ + dragontcp-server (Go, on the host) + │ + │ plain TCP + ▼ + destination server ``` -The important architectural decision is that the **server is only a relay**. It -does not create a TUN device, does not do NAT, and needs no `iptables` rules. -All the packet-level work happens on the phone, in userspace. This keeps the -server trivial to deploy (a single static binary) and keeps the phone side -independent of what the server can do. +### 1.2 Design decisions -### Why records instead of a raw stream +**The server is only a relay.** It creates no TUN device, performs no NAT, and +needs no `iptables` rules. All packet-level work happens on the phone in +userspace. The server is a single static binary with no dependencies. -A plain TCP tunnel sends a continuous byte stream. DragonTCP instead splits each -direction into independent **records**, each carried by its own request/response -exchange. That costs a little efficiency and buys two things: +**Each direction is split into records, not a byte stream.** Every record is +carried by its own request/response exchange. That costs some efficiency and buys +two properties: -1. **Record size is negotiable at runtime.** Some networks silently drop or - truncate large writes on port 53. Because every record is framed and - acknowledged, the client can discover the largest size that survives the path - and adapt when conditions change. -2. **The transport survives connection rotation.** Session state lives in a - 16-byte session ID, not in the TCP connection. The client can close and - reopen the underlying TCP connection between any two records without losing - the stream, which matters on middleboxes that cap how long a port-53 - connection may live or how many requests it may carry. +* *Record size is negotiable at runtime.* Some networks silently drop or truncate + large writes on port 53. Because every record is framed and acknowledged, the + client can discover the largest size that survives the path, and adapt if + conditions change later. +* *The transport survives connection rotation.* Session state lives in a 16-byte + session ID, not in the TCP connection. The client can close and reopen the + underlying connection between any two records without losing the stream, which + matters on middleboxes that limit how long a port-53 connection may live or how + many requests it may carry. + +**Stream position is explicit.** Both directions are addressed by absolute byte +offset rather than by message counter. That makes retries idempotent and makes +flow control a matter of reporting a number. --- -## 2. Security model — read this +## 2. Security model -**DragonTCP does not provide authenticated encryption. Do not treat it as a VPN -in the security sense.** +**DragonTCP does not provide authenticated encryption. It is not a VPN in the +security sense.** -What it actually does to payload bytes is **masking**: every payload is XORed -with a keystream derived from SHA-256. The keystream varies with session ID, -mode, sequence, direction, and block number, so the same plaintext does not -produce the same ciphertext twice, and there are no fixed ASCII markers such as -`UP`, `OK`, `CPUSH`, or `CPULL` on the wire. +Payloads are **masked**: XORed with a keystream derived from SHA-256 that varies +with session ID, mode, sequence, direction, and block number. The same plaintext +therefore does not produce the same ciphertext twice, and no fixed ASCII markers +appear on the wire. -That defeats trivial pattern matching. It does **not** defeat an adversary who -can read the traffic, because: +This defeats trivial pattern matching. It does not defeat anyone who can read the +traffic: -* the mask is derived from the **session ID, which is sent in cleartext in every - request header**. Anyone who sees the header can regenerate the keystream and - recover the plaintext. This is obfuscation, not confidentiality. -* record headers (mode, session, sequence, length) are never masked, -* `StatusError` bodies are sent **unmasked**, in plain text, -* there is no integrity check, so a network attacker can tamper with payloads - undetected. +* The mask is derived from the **session ID, which is transmitted in cleartext in + every request header.** Anyone who sees the header can regenerate the keystream + and recover the plaintext. This is obfuscation, not confidentiality. +* Record headers — mode, session, sequence, length — are never masked. +* `StatusError` bodies are sent **unmasked**, as plain text. +* There is no integrity check, so payloads can be tampered with undetected. -The optional `--token` is a shared secret compared in constant time. It gates -who may open sessions. It is not a key — it does not affect the mask. +The optional `--token` is a shared secret compared in constant time. It gates who +may open sessions. It is not a key and does not affect the mask. -**Practical consequence:** keep using TLS end to end. HTTPS through DragonTCP is -protected by HTTPS, not by DragonTCP. Never send plaintext credentials through a -plain-HTTP site over this tunnel and assume they are private. +**Consequence:** rely on TLS end to end. HTTPS through DragonTCP is protected by +HTTPS, not by DragonTCP. Do not assume anything sent over plain HTTP through this +tunnel is private. + +On the server side, the relay refuses private and special-use destinations by +default (§6.5). That restriction is what stops the tunnel being used to reach the +host's own localhost services or cloud metadata endpoints. --- @@ -124,28 +122,37 @@ plain-HTTP site over this tunnel and assume they are private. ```text core/ - go.mod module "dragontcp", Go 1.22, zero dependencies + go.mod module "dragontcp", Go 1.22, no dependencies cmd/dragontcp-client/ main.go local HTTP/HTTPS proxy, CLI flags - chunk.go the client transport: lanes, probing, adaptation - chunk_test.go adaptive sizer + reconnect tests + chunk.go transport: lanes, probing, adaptation, batching + chunk_test.go cmd/dragontcp-server/ main.go listener, DNS cache, address filtering, CLI flags chunk.go session manager, buffering, request dispatch - debug.go optional counters and periodic statistics + debug.go counters and periodic statistics chunk_test.go internal/wire/ - protocol.go the record format and the masking keystream - protocol_test.go mask round-trip and sequence-variance test + protocol.go record format and the masking keystream + protocol_test.go internal/protocol/ - protocol.go TCP tuning, relays, legacy UP/OK framing + protocol.go TCP tuning, relay loops, legacy UP/OK framing xor_*.go word-at-a-time XOR helpers (legacy path) android/ AndroidManifest.xml package com.dragontcp.client, minSdk 29 src/com/dragontcp/client/ Java: UI, VpnService, log screen - src/tech/xvanturing/freeproxy/ Kotlin: userspace TCP/IP stack (Apache 2.0, see §16) - res/ icon + theme + MainActivity.java settings screen + DragonService.java VpnService: spawns the core, owns the TUN + LogActivity.java live log viewer + AppLog.java in-memory log buffer + src/tech/xvanturing/freeproxy/ Kotlin: userspace TCP/IP stack (Apache 2.0, §14) + vpn/TunnelEngine.kt TUN read/write loops, session tables + vpn/TcpSession.kt userspace TCP endpoint + vpn/UdpSession.kt UDP/DNS handling + vpn/net/ IP/TCP/UDP headers, checksums, packet builder, DNS + vpn/proxy/ProxyClient.kt HTTP CONNECT client + res/ icon and theme assets/ license texts shipped inside the APK lib/arm64-v8a/libdragontcp_client.so the Go client, packaged as a native lib build_apk.ps1 / build_apk.cmd Windows build @@ -153,47 +160,54 @@ android/ build_core.ps1 / build_core.sh Go builds (Windows / Unix) build_all.sh core + APK in one step (Unix) -bin/ built server binaries +bin/ built server and client binaries licenses/ full Apache 2.0 text -THIRD_PARTY_NOTICES.md upstream attribution (required — do not delete) +SHA256SUMS digests for the built binaries +THIRD_PARTY_NOTICES.md upstream attribution (a license condition) ``` -Note that `core/internal/protocol` still contains the **legacy** `UP`/`OK` text -framing and the fixed `0xAD` XOR. That code is retained because -`internal/protocol` also holds the TCP tuning helpers and relay loops that the -current transport uses. The legacy framing itself is unreachable in normal -operation: `dragontcp-client` refuses to start unless `--transport chunk`. +There is exactly one copy of every source file. A second `core/`, or a nested +`android/android/`, is stale — `build_apk.ps1` guards against this by locating +the repo root as the nearest ancestor holding **both** `core/go.mod` and +`build_core.ps1`. + +`core/internal/protocol` still contains the legacy `UP`/`OK` text framing and a +fixed `0xAD` XOR. That package is retained because it also holds the TCP tuning +helpers and relay loops the current transport uses. The legacy framing itself is +unreachable: `dragontcp-client` refuses to start unless `--transport chunk`. --- ## 4. The wire protocol -Everything below is implemented in `core/internal/wire/protocol.go`. +Implemented in `core/internal/wire/protocol.go`. ### 4.1 Record framing -Every client-to-server message is a **request**: +Client to server — a **request**: ```text offset size field 0 1 mode 1 16 session ID - 17 8 sequence (big-endian uint64) - 25 4 payload length (big-endian uint32) - 29 n payload (masked) + 17 8 sequence (big-endian uint64) + 25 4 payload length (big-endian uint32) + 29 n payload (masked) ``` -Every server-to-client message is a **response**: +Server to client — a **response**: ```text offset size field 0 1 status - 1 4 body length (big-endian uint32) - 5 n body (masked, except where noted) + 1 4 body length (big-endian uint32) + 5 n body (masked, with exceptions in §4.3) ``` -Header sizes are therefore **29 bytes** and **5 bytes**. The hard payload ceiling -in the wire layer is 2 MiB (`MaxPayload`); the transport never exceeds 1 MiB. +Headers are **29 bytes** and **5 bytes**. The wire layer rejects payloads above +2 MiB (`MaxPayload`); the transport never exceeds 1 MiB. + +A single request may be answered by **several** responses — see `ModeDownload`. ### 4.2 Modes and statuses @@ -208,60 +222,60 @@ in the wire layer is 2 MiB (`MaxPayload`); the transport never exceeds 1 MiB. | Status | Value | Meaning | |---|---|---| | `StatusOK` | 0 | Success; body may carry a result | -| `StatusError` | 1 | Failure; body is a **plaintext** message | +| `StatusError` | 1 | Failure; body is a plaintext message | | `StatusData` | 2 | Body is stream data | -| `StatusWait` | 3 | Nothing available yet; poll again | +| `StatusWait` | 3 | Nothing available yet; ask again | | `StatusEOF` | 4 | Target closed the stream | -### 4.3 The masking keystream +### 4.3 Payload masking -```go +```text seed[0:16] = session ID seed[16] = mode -seed[17:25] = sequence (big-endian) +seed[17:25] = sequence (big-endian) seed[25] = 1 for responses, 0 for requests -seed[26:30] = block counter (big-endian, increments every 32 bytes) +seed[26:30] = block counter (big-endian, increments every 32 bytes) -keystream_block[i] = SHA256(seed) +keystream_block = SHA256(seed) payload ^= keystream ``` -Masking is its own inverse, so the same call encodes and decodes. Because the -sequence field is the **byte offset within the stream** (see below), consecutive -records never reuse a keystream position, and retransmitting the same offset -reproduces the same bytes — which is what makes idempotent retries safe. +Masking is its own inverse, so one function both encodes and decodes. Because the +sequence field is a **byte offset** (§4.5), consecutive records never reuse a +keystream position, and re-sending the same offset reproduces the same bytes — +which is what makes idempotent retries safe. -Not everything is masked. `WriteResponse` sends the body unmasked and is used for -`StatusOK` with no body, `StatusWait`, `StatusEOF`, and all `StatusError` -messages. `WriteMaskedResponse` is used for `StatusData` and for the `OPEN` -result. On the client, `DecodeMaskedResponse` deliberately skips decoding when -the status is `StatusError`, so the two sides agree. +Not everything is masked. `WriteResponse` sends the body as-is and is used for +empty `StatusOK`, `StatusWait`, `StatusEOF`, and every `StatusError`. +`WriteMaskedResponse` is used for `StatusData` and for the `OPEN` result. On the +client, `DecodeMaskedResponse` deliberately skips decoding when the status is +`StatusError`, so both sides agree. -### 4.4 Payload layouts per mode +### 4.4 Payload layouts -**PROBE** (`ModeProbe`) — request payload: +**PROBE** request payload: ```text offset size field 0 4 magic "DTP2" 4 1 probe kind - 5 2 token length (big-endian uint16) - 7 4 value (big-endian uint32) + 5 2 token length (big-endian uint16) + 7 4 value (big-endian uint32) 11 t token 11+t … filler, byte i = (i*31 + 17) & 0xFF ``` | Probe kind | Value | Server behaviour | |---|---|---| -| `ProbeUpload` | 1 | Replies `OK` if the whole record was received and is within `--chunk-max`. The *filler* is the thing being measured. | +| `ProbeUpload` | 1 | Replies `OK` if the whole record arrived and is within `--chunk-max`. The *filler* is what is being measured. | | `ProbeDownload` | 2 | Replies `StatusData` with exactly `value` bytes of the same generated pattern. | -| `ProbeKeepalive` | 3 | Replies `OK`. Used to test whether a connection may carry several requests. | +| `ProbeKeepalive` | 3 | Replies `OK`. Used to test whether one connection may carry several requests. | | `ProbeBatch` | 4 | Replies with up to 16 back-to-back 32-byte `StatusData` records. | The client verifies download probes byte for byte, so a middlebox that truncates -or rewrites the response fails the probe rather than silently corrupting data. +or rewrites the response fails the probe instead of silently corrupting data. -**OPEN** (`ModeOpen`) — request payload: +**OPEN** request payload: ```text offset size field @@ -272,16 +286,15 @@ offset size field 6+t h target host (name or literal IP) ``` -Response is `StatusOK` with a masked 4-byte body: the server's `--chunk-max`. -The client immediately clamps its own maximum to that value. Re-sending `OPEN` -for an existing session is idempotent and just returns the same limit again. +The response is `StatusOK` with a masked 4-byte body: the server's `--chunk-max`. +The client clamps its own maximum to that value. Repeating `OPEN` for an existing +session is idempotent and returns the same limit. -**UPLOAD** (`ModeUpload`) — the sequence field is the **byte offset in the upload -stream**, and the payload is the data. The server requires `offset` to equal -exactly what it expects next. Response is an empty `StatusOK` ACK. +**UPLOAD** — the sequence field is the byte offset in the upload stream and the +payload is the data. The response is an empty `StatusOK` acknowledgement. -**DOWNLOAD** (`ModeDownload`) — the sequence field is the **byte offset the -client wants next**. The 14-byte payload is: +**DOWNLOAD** — the sequence field is the byte offset the client wants next. The +14-byte payload is: ```text offset size field @@ -290,108 +303,118 @@ offset size field 12 2 how many records the client will accept in this batch ``` -The server replies with a *stream* of responses to that single request: up to -`count` `StatusData` records, each masked with the running offset, terminated -early by a single `StatusWait` or `StatusEOF`. This is the batching mechanism — -one request, many records. +The server answers with up to `count` `StatusData` records, each masked with the +running offset, terminated early by one `StatusWait` or `StatusEOF`. -**CLOSE** (`ModeClose`) — no payload; the server drops the session and replies -`StatusOK`. +**CLOSE** — no payload; the server drops the session and replies `StatusOK`. -### 4.5 A complete session, end to end +### 4.5 Stream offsets and flow control + +Three offsets drive the whole protocol: + +| Offset | Held by | Meaning | +|---|---|---| +| upload sequence | client | bytes already sent toward the target | +| download sequence | client | next byte the client wants | +| ack offset | client, reported to server | next byte the **application** has not yet read | + +The ack offset trails the download offset. It advances only when the consuming +application actually reads bytes out of the client's buffer. The server uses it to +discard acknowledged data; when the client stops reading, the server's buffer +fills, its reader goroutine blocks, and TCP backpressure propagates to the origin +server. There is no separate window mechanism — the ack number is the window. + +### 4.6 A session end to end ```text client server - |-- PROBE upload (binary search) ---------->| - |<- OK / error ---------------------------------| - |-- PROBE download (binary search) ---------->| - |<- DATA(pattern) ------------------------------| - |-- PROBE keepalive x8 on one connection ----->| - |<- OK x8 -------------------------------------| - | - |-- OPEN sid=… host=example.com port=443 ------>| dial example.com:443 - |<- OK body=chunk_max --------------------------| - | - |-- UPLOAD sid seq=0 payload=TLS ClientHello>| write() to target - |<- OK ----------------------------------------| - |-- DOWNLOAD sid seq=0 ack=0 limit=1400 count=4>| - |<- DATA(1400) DATA(1400) DATA(900) WAIT --------| - |-- UPLOAD sid seq=517 payload=… ------------->| - |<- OK ----------------------------------------| - |-- DOWNLOAD sid seq=3700 ack=3700 … ---------->| - |<- EOF ----------------------------------------| - |-- CLOSE sid ---------------------------------->| - |<- OK ----------------------------------------| + │── PROBE upload (binary search) ─────────▶│ + │◀─ OK / error ────────────────────────────────│ + │── PROBE download (binary search) ─────────▶│ + │◀─ DATA(pattern) ─────────────────────────────│ + │── PROBE keepalive ×8 on one connection ────▶│ + │◀─ OK ×8 ─────────────────────────────────────│ + │ + │── OPEN sid=… host=example.com port=443 ─────▶│ dial example.com:443 + │◀─ OK body=chunk_max ─────────────────────────│ + │ + │── UPLOAD sid seq=0 payload=ClientHello ─────▶│ write() to target + │◀─ OK ────────────────────────────────────────│ + │── DOWNLOAD sid seq=0 ack=0 limit=1400 cnt=4 ▶│ + │◀─ DATA(1400) DATA(1400) DATA(900) WAIT ──────│ + │── UPLOAD sid seq=517 payload=… ─────────────▶│ + │◀─ OK ────────────────────────────────────────│ + │── DOWNLOAD sid seq=3700 ack=3700 … ─────────▶│ + │◀─ EOF ───────────────────────────────────────│ + │── CLOSE sid ────────────────────────────────▶│ + │◀─ OK ────────────────────────────────────────│ ``` -Note `ack` trailing behind `seq`: the client advances `ack` only when the -application has actually read the bytes, which is what applies backpressure all -the way to the origin server. - --- ## 5. The Go client Source: `core/cmd/dragontcp-client/`. -### 5.1 The local proxy front end (`main.go`) +### 5.1 Local proxy front end -The client listens on `127.0.0.1:8080` and speaks ordinary HTTP proxy protocol: +`main.go` listens on `127.0.0.1:8080` and speaks ordinary HTTP proxy protocol. -* **`CONNECT host:port`** — opens a tunnel, replies `200 Connection - Established`, then relays bytes in both directions. This is the path used for - HTTPS and, on Android, for everything. -* **Plain `GET http://…`** — the request line is rewritten to origin form, the - `Connection`, `Proxy-Connection`, and `Proxy-Authorization` headers are - stripped, a `Host` header is synthesised if missing, and `Connection: close` - is appended. +**`CONNECT host:port`** opens a tunnel, replies `200 Connection Established`, and +relays bytes both ways. This is the path used for HTTPS and, on Android, for +everything. -Accepts are bounded by `--max-connections` (default 20 000) using a slot -channel; over the limit the client returns `503`. Relaying uses `io.Copy` in both -directions and waits for **both** directions to finish, preserving TCP half-close -so large or slow responses are not truncated. +**Plain `GET http://…`** is rewritten to origin form: the request line is +rebuilt, `Connection`, `Proxy-Connection`, and `Proxy-Authorization` are stripped, +a `Host` header is synthesised if absent, and `Connection: close` is appended. -### 5.2 `chunkConn` — a stream that looks like a socket +Request headers are read until `\r\n\r\n`, with a 128 KiB ceiling. The local +connection carries a 15-second deadline during the handshake, cleared once +relaying starts. Accepts are bounded by `--max-connections` through a slot +channel; over the limit the client returns `503`. -`openChunkTunnel` returns a `chunkConn` that implements `net.Conn`, so the proxy -front end does not know it is talking to a record protocol. Internally it keeps: +Relaying uses `io.Copy` in both directions and waits for **both** to finish, so +TCP half-close is preserved and slow or large responses are not truncated. On +Linux this lets the kernel use `splice`. -* `upOffset` — bytes sent so far; used as the upload sequence, -* `downloadOffset` — the next byte the client will ask for, -* `consumedOffset` — the next byte the application has not yet read; sent as - `ack`, -* `readBuf` — data received but not yet handed to the reader, +### 5.2 `chunkConn` + +`openChunkTunnel` returns a `chunkConn` implementing `net.Conn`, so the proxy +front end never knows it is talking to a record protocol. It holds: + +* `upOffset` — bytes sent, used as the upload sequence, +* `downloadOffset` — next byte to request, +* `consumedOffset` — next unread byte, sent as `ack`, +* `readBuf` — received but not yet delivered to the reader, * two independent `requestLane`s, one for uploads and one for downloads, so a blocking download poll never delays an upload. -`Write` slices the caller's buffer into records of the current upload size and -sends them one at a time, each acknowledged before the next. `Read` refills -`readBuf` through `fillReadBuffer`, which issues batched download requests. +`Write` slices the caller's buffer into records of the current upload size, each +acknowledged before the next is sent. `Read` refills `readBuf` via +`fillReadBuffer`, which issues batched download requests. `Close` sends +`ModeClose` on a throwaway lane and closes both persistent lanes. ### 5.3 Request lanes and connection reuse A `requestLane` owns at most one physical TCP connection and serialises requests -onto it with a mutex. `reconnectEvery` controls rotation: +onto it under a mutex. Any I/O error discards the connection; the next request +redials. Sockets get `TCP_NODELAY`, 30-second keepalives, and optionally explicit +buffer sizes via `--tcp-buffer`. | `--chunk-reconnect-every` | Behaviour | |---|---| -| `0` | Persistent. Keep one connection for the life of the lane. | -| `1` | Auto. The path probe decides: persistent if reuse worked, otherwise one logical request per TCP connection. | -| `N ≥ 2` | Rotate: close and redial after N logical requests. | - -Any I/O error discards the connection immediately; the next request redials. All -sockets get `TCP_NODELAY` and 30-second keepalives, and optionally explicit -socket buffer sizes via `--tcp-buffer` (0 leaves OS autotuning alone, which is -the right default). +| `0` | Persistent — one connection for the life of the lane | +| `1` | Auto — persistent if the path probe showed reuse works, otherwise one logical request per connection | +| `N ≥ 2` | Rotate — close and redial after N logical requests | ### 5.4 Path probing Before the first real connection, `getPathProfile` measures the path once and -caches the result for **30 minutes**, keyed by server address, token, and the -size bounds. +caches the result for **30 minutes**, keyed by server address, token, and size +bounds. -Upload and download are probed **independently and concurrently**, each by -binary search over a fixed ladder of candidate sizes: +Upload and download are probed **independently and concurrently**, each by binary +search over a fixed ladder: ```text 32, 64, 128, 256, 512, 1024, 1200, 1280, 1320, 1350, 1360, 1380, 1400, @@ -399,17 +422,16 @@ binary search over a fixed ladder of candidate sizes: 131072, 262144, 524288, 786432, 1048576 ``` -The ladder is filtered to `[--chunk-min, --chunk-max]`, and the configured bounds -are added if missing. Binary search means roughly five probes instead of -twenty-eight, and — critically — it means the client does not have to *fail* at -every size on the way down during real traffic. +The ladder is filtered to `[--chunk-min, --chunk-max]`, with the configured bounds +added if missing. Binary search costs roughly five probes instead of twenty-eight, +and avoids having to *fail* at every size on the way down during real traffic. -A third probe sends eight keepalives on a single connection to decide whether -request reuse survives the path. Each probe uses a fresh random session ID and a -timeout capped at 2.5 s; if the whole search does not finish within 20 s, the -client falls back to 32 768 up / 1 350 down. +A third probe sends eight keepalives on one connection to decide whether request +reuse survives. Each probe uses a fresh random session ID with a timeout capped at +2.5 s. If the search does not finish within 20 s, the client falls back to +32 768 up / 1 350 down. -The result is logged once: +The outcome is logged once: ```text path probe: upload=32768 download=1400 persistent=true @@ -417,29 +439,26 @@ path probe: upload=32768 download=1400 persistent=true ### 5.5 Adaptive record sizing -Runtime adaptation remains as a safety net after probing, in `adaptiveSizer`. -Each direction keeps its own instance plus two landmarks: `good` (largest size -known to work) and `bad` (smallest size known to fail). +`adaptiveSizer` keeps one instance per direction plus two landmarks: `good`, the +largest size known to work, and `bad`, the smallest known to fail. **On failure** at the current size: * record `bad = min(bad, attempted)`, * drop to `good` if a smaller known-good size exists, otherwise halve, -* clamp to `--chunk-min`, and force strict decrease. +* clamp to `--chunk-min` and force a strict decrease. **On success** at the current size, after `--chunk-grow-after` consecutive successes (default 16): -* if a `bad` landmark is known and is more than one step above, move **halfway - toward it** — a binary search upward rather than a blind jump, +* if `bad` is known and more than one step above, move **halfway toward it** — a + binary search upward rather than a blind jump, * otherwise clear the stale `bad` landmark and grow by `max(current/4, 32)`, * clamp to the maximum. -If `bad - good ≤ 64` the required success count is multiplied by eight: once the -working size is bracketed tightly, the controller stops probing the ceiling -aggressively and settles. - -Changes are logged: +When `bad - good ≤ 64` the required success count is multiplied by eight: once the +working size is tightly bracketed the controller stops probing the ceiling and +settles. ```text adaptive upload chunk: 32768 -> 16384 after transport failure @@ -447,32 +466,62 @@ adaptive download chunk: 1400 -> 700 after transport failure adaptive upload chunk: 700 -> 1050 after stable success ``` -### 5.6 Download batching and pipeline depth +### 5.6 Download batching -One download request can return many records. The batch size is -`--chunk-concurrency` (1–256, default 1), additionally capped so that one batch -carries roughly 1 MiB of useful data: +One download request can return many records. **This is a transport setting, not +a thread count** — nothing here creates threads. It controls how many records the +server may stream back in reply to a single request. + +Two bounds define it: + +| Flag | Default | Meaning | +|---|---|---| +| `--chunk-concurrency` | `1` | Maximum records per request (1–256) | +| `--chunk-concurrency-min` | `1` | Minimum records per request (1–256) | + +Batching matters most on restricted paths. If the safe record size is 32 bytes, +one TCP/53 request can still return many 32-byte records instead of needing a +fresh request for every 32 useful bytes. + +**Two modes**, decided entirely by whether the bounds are equal: + +* **Pinned** (`min == max`). The depth never changes: it does not grow, it does + not halve on failure, and it is not reduced by the 1 MiB batch cap. Use this + when a path only works at one specific number of records — the adaptive + controller stays completely out of the way. +* **Adaptive** (`min < max`). The depth starts at the ceiling, **halves** on + transport failure but never below `min`, and **grows by one** after successful + data responses until it reaches the ceiling again. + +`1..1` is the pinned case at one record, which is the default. + +```text +adaptive download batch: 64 -> 32 after transport failure +``` + +Outside pinned mode, each batch is additionally capped to carry roughly 1 MiB of +useful data: ```go count = min(pipeline, maxPipeline, (1 MiB) / chunkSize) +count = max(count, minPipeline) // the floor always wins ``` -This matters most on restricted paths. If the safe record size is 32 bytes, one -TCP/53 request can still return many 32-byte records instead of needing a fresh -request for every 32 useful bytes. +The server independently clamps `count` to 256 and `limit` to its own +`--chunk-max`, so a client can never demand more than the server allows. -Depth adapts within `1..N`: it starts at the ceiling, **halves** on transport -failure, and **grows by one** after successful data responses. A ceiling of 1 -disables the mechanism and stays fixed at 1. +### 5.7 Failure escalation -```text -adaptive download pipeline: 64 -> 32 after transport failure -``` +On a download failure the client escalates in a fixed order: -The escalation order on repeated failure is deliberate: shrink the pipeline -first, and only when depth is already 1 start shrinking the record size. Eight -consecutive failures at the minimum record size abort the connection with an -error rather than spinning forever. +1. **Shrink the batch** — halve it, never below the floor. +2. **Shrink the record size** — only once the batch is already at its floor. +3. **Give up** — after eight consecutive failures at the minimum record size, + return an error rather than spinning forever. + +Uploads have no batch dimension, so they go straight to steps 2 and 3. A 30 ms +pause separates retries. `StatusWait` responses are followed by a +`--chunk-poll-delay` pause (default 2 ms) instead of being treated as failures. --- @@ -482,80 +531,92 @@ Source: `core/cmd/dragontcp-server/`. ### 6.1 Connection handling -The server accepts on `0.0.0.0:53` by default, bounded by `--max-connections`. -Each connection runs a loop: read one request (30-second deadline), dispatch it, -repeat. Because session state is keyed by session ID rather than by connection, -requests for one logical stream may arrive over many connections, in any order -the client chooses. +The server listens on `0.0.0.0:53` by default, bounded by `--max-connections` +through a slot channel. Each connection runs a loop: read one request with a +30-second deadline, dispatch it, repeat. Because session state is keyed by session +ID rather than by connection, requests for one logical stream may arrive over many +connections in whatever pattern the client chooses. -### 6.2 Session state +### 6.2 Session state and buffering Each `OPEN` creates a `streamSession` holding the real TCP connection to the -target plus a **download buffer**: +target plus a download buffer: -* `buf` holds bytes that have arrived from the target but are not yet - acknowledged by the client, -* `base` is the absolute stream offset of `buf[0]`, -* a dedicated goroutine reads the target in 64 KiB chunks and appends to `buf`. +* `buf` — bytes received from the target but not yet acknowledged, +* `base` — the absolute stream offset of `buf[0]`, +* a dedicated goroutine reading the target in 64 KiB chunks. -That goroutine **blocks when the buffer is full**, which is the whole flow -control story: a slow phone stops draining, `ack` stops advancing, the buffer -fills, the server stops reading, and TCP backpressure propagates to the origin -server. Buffer size is `--chunk-buffered × 65536`, clamped to 1 MiB…64 MiB -(default 256 → 16 MiB per session). +That goroutine **blocks when the buffer is full**, which is the entire flow +control story described in §4.5. Buffer size is `--chunk-buffered × 65536`, +clamped to 1 MiB…64 MiB (default 256 → 16 MiB per session). -`ack` drops acknowledged bytes off the front and advances `base`. The buffer is -compacted when its capacity grows past four times its length and exceeds 1 MiB, -so long-lived sessions do not hold onto peak allocations. +Acknowledgement drops bytes off the front and advances `base`. The buffer is +compacted when its capacity exceeds four times its length and is over 1 MiB, so +long-lived sessions do not hold onto peak allocations. -### 6.3 Serving a download +Sessions are stored in a map keyed by the raw 16 session bytes. Creating a session +that already exists closes the newcomer and keeps the original. -`readAt(offset, limit, wait)` enforces that `offset` is within -`[base, base+len(buf)]` — a request below `base` is an error, because those bytes -were already acknowledged and discarded. +### 6.3 Serving downloads -Two behaviours are worth knowing: +`readAt(offset, limit, wait)` requires `offset` to lie within +`[base, base+len(buf)]`. An offset below `base` is an error — those bytes were +acknowledged and discarded. An offset beyond the buffered end is also an error. + +Two behaviours matter: * **Long poll.** The *first* record of a batch waits up to `--chunk-poll-wait` - (default 200 ms) for data. Later records in the same batch do not wait: the - batch drains whatever is buffered and then returns `StatusWait`. This keeps - batches from stalling on partially-filled pipelines. -* **Coalescing.** If less than `limit` bytes are available, the server waits up + (default 200 ms) for data to arrive. Later records in the same batch do not + wait: the batch drains what is buffered and then returns `StatusWait`. This + stops batches stalling on partially-filled pipelines. +* **Coalescing.** If fewer than `limit` bytes are available, the server waits up to 2 ms more for the target to produce more. Without this, a 1-byte read from - the origin would become a permanent 1-byte tunnel record, and the per-record - overhead would dominate. + the origin would become a permanent 1-byte tunnel record and per-record overhead + would dominate. -### 6.4 Serving an upload +### 6.4 Serving uploads -Uploads must arrive in exact order: `offset` must equal the session's -`expectedUp`. Two cases are special-cased: +Uploads must arrive in exact order: the offset must equal the session's +`expectedUp`. Two cases are special: -* an offset entirely **below** `expectedUp` is treated as an idempotent retry - after a lost ACK and silently succeeds, +* an offset entirely **below** `expectedUp` is an idempotent retry after a lost + acknowledgement and silently succeeds, * a partially overlapping retry is rejected as an error. This is what makes it safe for the client to resend a record whose response was lost when a connection died mid-request. -### 6.5 Safety and lifecycle +### 6.5 Target resolution and address filtering -* **Token.** Compared with `crypto/subtle.ConstantTimeCompare` on both `PROBE` - and `OPEN`. An empty token means no authentication. -* **Target filtering.** By default the server refuses to dial unspecified, - multicast, private, loopback, link-local, and a list of special-use prefixes - (`0.0.0.0/8`, `100.64.0.0/10`, `192.0.2.0/24`, `198.18.0.0/15`, `240.0.0.0/4`, - `2001:db8::/32`, and others). `--allow-private` disables this. **Leave it off - on a public server** — it is what stops the tunnel being used to reach your - VPS's own localhost services and cloud metadata endpoints. -* **DNS cache.** Bounded map with a TTL (`--dns-cache-ttl`, default 30 s; - `--dns-cache-size`, default 4096). When full it resets wholesale rather than - evicting entry by entry — cheap, and adequate for a hot cache. -* **Idle reaping.** A sweep every 30 s closes sessions idle longer than - `--chunk-session-timeout` (default 2 minutes). -* **Debug.** `--debug` logs accepts, session opens, and errors to stderr, and - `--debug-stats-interval` prints counters (bytes up/down, push records, pull - requests, data/wait records, active sessions). `--debug-chunks` logs every - record and is very verbose. +`dialTarget` resolves through a bounded DNS cache (`--dns-cache-ttl`, default +30 s; `--dns-cache-size`, default 4096). When the cache is full it resets +wholesale rather than evicting entry by entry — cheap, and adequate for a hot +cache. Literal IPs bypass resolution. + +Each candidate address is checked before dialling. Rejected by default: +unspecified, multicast, non-global-unicast, private, loopback, link-local, and +these special-use prefixes: + +```text +0.0.0.0/8 100.64.0.0/10 192.0.0.0/24 192.0.2.0/24 +198.18.0.0/15 198.51.100.0/24 203.0.113.0/24 240.0.0.0/4 +2001:db8::/32 +``` + +`--allow-private` disables the whole check. Leave it off on a public server. + +If every resolved address is blocked, the error names them. Dial timeout is 10 s, +with 30-second keepalives on the resulting connection. + +### 6.6 Lifecycle and diagnostics + +A sweep every 30 s closes sessions idle longer than `--chunk-session-timeout` +(default 2 minutes), and any session already marked closed. + +`--debug` logs accepts, session opens, and errors to stderr. +`--debug-stats-interval` prints counters: bytes up and down, push records, pull +requests, data and wait records, sessions opened and closed, active sessions. +`--debug-chunks` logs every record and is very verbose. --- @@ -563,83 +624,172 @@ lost when a connection died mid-request. Package `com.dragontcp.client`, `minSdk 29`, `targetSdk 29`, arm64 only. -### 7.1 Process model +### 7.1 Components -The APK ships the Go client at `lib/arm64-v8a/libdragontcp_client.so`. Despite -the name it is not a shared library — it is a **statically linked Go -executable**. The `lib*.so` naming and `extractNativeLibs="true"` make Android -unpack it into `nativeLibraryDir` with the executable bit set, which is the -standard way to ship a helper binary in an APK without needing an installer. +| Component | Language | Role | +|---|---|---| +| `MainActivity` | Java | Settings screen; validates and persists configuration | +| `DragonService` | Java | `VpnService`: spawns the Go core, owns the TUN, runs the foreground notification | +| `LogActivity` / `AppLog` | Java | Live log viewer over a 600-line in-memory ring | +| `TunnelEngine` | Kotlin | TUN read/write loops, session tables, housekeeping | +| `TcpSession` | Kotlin | Userspace TCP endpoint, one per 4-tuple | +| `UdpSession` | Kotlin | UDP handling; in practice DNS only | +| `net/*` | Kotlin | Header parsing, checksums, packet construction, DNS parsing | +| `ProxyClient` | Kotlin | HTTP CONNECT client against the local Go proxy | -`DragonService` launches it with `ProcessBuilder`, merges stderr into stdout, -and reads its output on a background thread. Only interesting lines reach the -UI log: those starting with `adaptive ` or `path probe:`, and anything -containing `error` or `failed`. A watchdog thread waits on the process; if the -core exits while the tunnel is supposed to be up, the whole VPN is torn down. +### 7.2 Process model -### 7.2 Startup sequence +The APK ships the Go client at `lib/arm64-v8a/libdragontcp_client.so`. Despite the +name it is not a shared library — it is a statically linked Go executable. The +`lib*.so` naming plus `extractNativeLibs="true"` makes Android unpack it into +`nativeLibraryDir` with the executable bit set, which is the standard way to ship +a helper binary in an APK. + +`DragonService` launches it with `ProcessBuilder`, merges stderr into stdout, and +reads the output on a background thread. Only interesting lines reach the UI log: +those beginning with `adaptive ` or `path probe:`, and anything containing `error` +or `failed`. A watchdog thread waits on the process; if the core exits while the +tunnel is supposed to be up, the whole VPN is torn down. + +### 7.3 Startup and shutdown + +Startup: 1. `MainActivity` validates the form and saves it to `SharedPreferences`. -2. `VpnService.prepare()` — the system consent dialog, if not already granted. +2. `VpnService.prepare()` shows the system consent dialog if needed. 3. `DragonService` starts in the foreground with a notification carrying a STOP action. 4. The Go core is spawned with flags built from the saved settings. -5. The service polls `127.0.0.1:8080` for up to 10 s until the proxy accepts. +5. The service polls `127.0.0.1:8080` for up to 10 s (150 ms connect timeout, + 100 ms between attempts) until the proxy accepts. 6. The TUN interface is established. -7. `TunnelEngine` starts and the state broadcasts flip to `CONNECTED`. +7. `TunnelEngine` starts; state broadcasts flip to `CONNECTED`. A service-side guard makes a duplicate CONNECT intent a no-op, so a double tap cannot tear down a healthy tunnel and restart it. -### 7.3 The TUN interface +Shutdown runs in a fixed order: stop the engine, close the TUN descriptor, then +`destroy()` the core process, waiting 1200 ms before `destroyForcibly()` and a +further 800 ms. `onRevoke()` (permission withdrawn by the system) and `onDestroy()` +both route through the same path. + +### 7.4 The TUN interface ```java setSession("DragonTCP Lite") setMtu(1400) -addAddress("10.77.0.2", 32) addRoute("0.0.0.0", 0) +addAddress("10.77.0.2", 32) addRoute("0.0.0.0", 0) addAddress("fd77:6472:6167:6f6e::2", 128) addRoute("::", 0) addDnsServer("1.1.1.1") addDisallowedApplication() -setBlocking(true) setMetered(false) +setBlocking(true) setMetered(false) ``` -Two decisions matter here: +Two decisions matter: * **IPv6 is captured, then dropped.** The userspace stack is IPv4-only. Routing - `::/0` into the tunnel and discarding it is what prevents apps from quietly - bypassing the tunnel over IPv6. It is a blackhole by design, not an oversight. + `::/0` into the tunnel and discarding it prevents apps from bypassing the tunnel + over IPv6. It is a blackhole by design. * **The app excludes itself** from the VPN, and every upstream socket is additionally passed through `VpnService.protect()`. Both are needed so the Go - core's connection to your VPS does not get routed back into the TUN it is - serving. + core's connection to the server is not routed back into the TUN it serves. -### 7.4 The userspace TCP/IP stack +### 7.5 Packet dispatch -`TunnelEngine` (Kotlin, adapted from FreeProxy — see §16) runs one reader thread -and one writer thread over the TUN file descriptor, plus a cached thread pool -exposed to coroutines for per-session blocking I/O. +`TunnelEngine` runs one reader thread and one writer thread on the TUN file +descriptor, plus a cached thread pool exposed to coroutines for per-session +blocking I/O. -* IPv4 packets are parsed; anything else (IPv6, fragments, ICMP) is dropped. -* **TCP** goes to a `TcpSession` keyed by the 4-tuple. The session acts as the - server endpoint towards the phone's own kernel: it answers SYN with SYN-ACK, - acknowledges data, and sends FIN/RST. Because the "link" to the kernel is - lossless, there is no congestion control — it only has to respect the peer's - advertised receive window. New flows are only created by a SYN; anything else - gets an RST so apps fail fast instead of hanging. Limit: 512 concurrent TCP - sessions. -* **UDP** goes to a `UdpSession`. With an HTTP CONNECT upstream, general UDP - cannot be carried, so **only DNS is handled**: queries are converted to - DNS-over-TCP (RFC 7766, 2-byte length prefix) and sent to **1.1.1.1:53** - through the tunnel. All other UDP is dropped, which makes QUIC fail and pushes - apps back to TCP. Limit: 256 sessions. -* Housekeeping every 5 s expires idle sessions (TCP 300 s, DNS 20 s, other UDP - 120 s). The TUN write queue holds 1024 packets and drops on overflow rather - than blocking session threads. +The reader parses IPv4 only. `Ipv4Header.parse` rejects anything that is not +version 4, has an inconsistent length, or is a **fragment** (MF set or a non-zero +fragment offset) — the stack does no reassembly, and since the MTU is chosen +locally, normal traffic never fragments. Non-IPv4, ICMP, and malformed packets are +dropped silently. -Real traffic reaches the Go proxy through `ProxyClient`, which opens a protected -socket to `127.0.0.1:8080` and issues `CONNECT :` per stream. +* **TCP** goes to a `TcpSession` keyed by the 4-tuple. New flows may only be + created by a SYN; anything else receives an RST so apps fail fast instead of + hanging. `putIfAbsent` prevents a retransmitted SYN from creating two sessions. +* **UDP** goes to a `UdpSession`, created on first packet. -### 7.5 Settings and how they map to flags +Limits: 512 concurrent TCP sessions, 256 UDP. The TUN write queue holds 1024 +packets and **drops** on overflow rather than blocking session threads — if the +kernel side cannot keep up, dropping is the correct behaviour for a link layer. +Housekeeping every 5 s expires idle sessions: TCP 300 s, DNS 20 s, other UDP 120 s. + +### 7.6 The userspace TCP endpoint + +`TcpSession` acts as the *server* toward the phone's own kernel: it answers SYN +with SYN-ACK, acknowledges data, and sends FIN or RST. The real traffic travels +through a socket to the local proxy. + +The key simplification: packets written to the TUN go to the local kernel over a +lossless path, so **no congestion control is required**. Respecting the peer's +advertised receive window is sufficient; retransmission exists only as a backstop. + +| Constant | Value | +|---|---| +| MSS | `MTU − 40`, clamped to 536…1460 (1360 at MTU 1400) | +| Receive window | 65535 | +| Max in flight | 65535 | +| Upstream queue | 64 chunks | +| Retransmit timeout | 400 ms | +| Window poll interval | 100 ms | + +**Connection setup.** On SYN the session records the peer's sequence and window, +then asynchronously opens the tunnel. If that fails it sends an RST immediately so +the app sees "connection refused" instead of waiting for a timeout. On success it +sends SYN-ACK carrying an MSS option, so the kernel never hands down a segment +larger than the tunnel MTU. The initial sequence number is random. + +**Inbound data.** Only in-order segments are accepted (`sequence == receiveNext`); +anything else is answered with a bare ACK. Accepted payloads are pushed to a +bounded channel. If that channel is full, `receiveNext` is *not* advanced and the +advertised window shrinks — eventually to zero — which pauses the application. +When the upstream pump drains a chunk and the window crosses back above one MSS, +a single window-update ACK is sent, avoiding a redundant ACK per chunk. + +**Outbound data.** The downstream pump reads MSS-sized buffers from the proxy +socket and writes them back as IPv4+TCP packets with PSH|ACK, throttled by +`awaitSendWindow`, which blocks until in-flight bytes plus the new chunk fit +inside `min(max(peerWindow, mss), 65535)`. Every segment is copied into a +retransmit queue; if no ACK arrives for 400 ms, the queue head is retransmitted. +Acknowledged segments are released from the front of the queue, with sequence +comparisons done as 32-bit signed differences so wraparound is handled correctly. + +**Teardown.** A FIN from the app closes the upstream channel; once the pump +drains it, `shutdownOutput()` tells the proxy the request is complete. When the +proxy side reaches EOF the session sends FIN; on an exception while established it +sends RST. `sendReset` uses its own buffer because it may run while another thread +holds the shared output buffer. + +### 7.7 UDP and DNS + +With an HTTP CONNECT upstream, general UDP cannot be carried. `UdpSession` +therefore handles **DNS only**; every other UDP flow is dropped, which makes QUIC +fail and pushes apps back to TCP. + +DNS queries are converted to **DNS-over-TCP** (RFC 7766: a 2-byte big-endian +length prefix followed by the message) and sent to **1.1.1.1:53** through the +tunnel. One tunnel is opened per query with a 10-second timeout, and the session +ends after the answer is written back to the TUN as a synthesised UDP datagram. + +Responses are also parsed to populate `HostRegistry`, a 512-entry access-ordered +LRU mapping IP → hostname. It exists so diagnostics can say `github.com:443` +rather than `140.82.121.4:443`. + +The stack retains hooks for direct (non-tunnelled) DNS, SOCKS5 UDP association, +DNS blocking, and per-app attribution. In this build `AppResolver`, `DnsBlocker`, +and `TunnelLog` are deliberate no-op stubs that preserve the upstream API. + +### 7.8 Packet construction + +`PacketBuilder` writes IPv4 packets into a caller-supplied buffer and returns the +length, allocating nothing on the hot path. Headers carry TTL 64, the Don't +Fragment flag, and a monotonically increasing identification field. Checksums +follow RFC 1071, with the TCP/UDP pseudo-header sum folded in; a computed UDP +checksum of zero is written as `0xFFFF` per RFC 768. + +### 7.9 Settings | UI field | Default | Flag passed to the core | |---|---|---| @@ -648,7 +798,8 @@ socket to `127.0.0.1:8080` and issues `CONNECT :` per stream. | Token | empty | `--token` (omitted entirely when blank) | | Max chunk | 1048576 | `--chunk-max` **and** `--chunk-start` | | Min chunk | 32 | `--chunk-min` | -| Concurrency | 1 | `--chunk-concurrency` | +| Batch max | 1 | `--chunk-concurrency` | +| Batch min | 1 | `--chunk-concurrency-min` | | Reconnect every | 0 | `--chunk-reconnect-every` | | Timeout (s) | 2 | `--chunk-timeout` | @@ -656,71 +807,63 @@ Fixed by the service: `--listen-host 127.0.0.1`, `--listen-port 8080`, `--transport chunk`, `--chunk-pollers 1`, `--chunk-grow-after 16`, `--chunk-adapt-log=true`. -In the current source, `Reconnect every` accepts `0` and `0` means persistent. -(Older *prebuilt* APKs shipped a Logs-page UI whose Reconnect field required at -least `1`; on those, `1` selects Auto. If you build from this source you get the -explicit `0 = persistent` behaviour.) +Settings are laid out in four cards: CONNECTION, RECORD SIZE, DOWNLOAD BATCH, and +ADVANCED. The batch card carries a live hint that restates the current setting in +words as you type, so the mode is never ambiguous: -`AppLog` keeps the last 600 lines in memory and pushes them live to -`LogActivity`. It is not persisted to disk. +```text +Batch max 1 Batch min 1 → Fixed: 1 record per request, never adapts. +Batch max 5 Batch min 5 → Pinned: exactly 5 records per request, never adapts. +Batch max 16 Batch min 1 → Adaptive: starts at 16, falls back toward 1 + on errors, recovers to 16. +Batch max 4 Batch min 9 → Batch min must not be greater than batch max. +``` + +The service independently clamps both values into 1–256 and forces `min ≤ max` +before spawning the core, and records the resulting mode in the log: + +```text +Download batch: pinned at 5 records per request (never adapts) +``` + +Validation ranges: port 1–65535, max chunk 32–1048576, min chunk 32–max chunk, +batch values 1–256 with `min ≤ max`, reconnect 0–1000000, timeout 1–120. + +### 7.10 Logs + +`AppLog` keeps the last 600 lines in memory and pushes them live to any +registered listener. `LogActivity` renders them in a selectable monospace view +with BACK and CLEAR actions and subscribes for live updates while visible. Nothing +is written to disk. --- -## 8. Building on Windows +## 8. Building -No Gradle and no Android Studio required. `android\build_apk.ps1` drives the -Android SDK command-line tools directly. +### 8.1 Windows -### 8.1 Quick start +No Gradle and no Android Studio required; `android\build_apk.ps1` drives the SDK +command-line tools directly. ```powershell cd android .\build_apk.ps1 ``` -Or double-click `android\build_apk.cmd`. The result is: +Or double-click `android\build_apk.cmd`. The result is +`android\build\DragonTCP-Hybrid-arm64.apk`, signed with a debug keystore +generated on first run. -```text -android\build\DragonTCP-Hybrid-arm64.apk -``` - -signed with a debug keystore that is generated on first run. - -### 8.2 Requirements - -| Component | How it is found | Needed? | +| Component | How it is found | Required? | |---|---|---| | **Android SDK** | `ANDROID_SDK_ROOT`, `ANDROID_HOME`, `%LOCALAPPDATA%\Android\Sdk`, `C:\Android\Sdk`, or `-SdkRoot` | Yes | -| **build-tools** | Newest installed version that has `aapt.exe`, `d8.bat`, `apksigner.bat`, `zipalign.exe`; or `-BuildTools` | Yes | -| **Platform** | `android-35` if present, else the newest with an `android.jar`; or `-Platform` | Yes | +| **build-tools** | Newest version having `aapt.exe`, `d8.bat`, `apksigner.bat`, `zipalign.exe`; or `-BuildTools` | Yes | +| **Platform** | `android-35` if present, else newest with an `android.jar`; or `-Platform` | Yes | | **JDK 17+** | `JAVA_HOME`, then `javac` on `PATH` (only if `jar.exe` sits beside it), then `C:\Program Files\Java\*`; or `-JavaHome` | Yes — a JRE is not enough | | **Kotlin** | `KOTLIN_HOME`, else `android\.tools\kotlinc-`; downloaded automatically if absent | Auto | | **Go** | `PATH`, or `-GoBin` on `build_core.ps1` | Only to rebuild the `.so` | -If the Kotlin compiler is missing, the script downloads it once (~85 MB) from -the JetBrains GitHub releases into `android\.tools\` and reuses it forever after. -Pass `-NoDownload` to make a missing Kotlin a hard error instead. - -### 8.3 What the script actually does - -1. **Resolve the toolchain** and print what it picked. -2. **Build the native core** if `lib\arm64-v8a\libdragontcp_client.so` is missing - (or `-BuildCore` was passed) by calling `..\build_core.ps1 -ClientOnly`. -3. **`aapt package`** — compile `res/`, pack `assets/`, bind the manifest against - `android.jar`, producing `resources.ap_`. -4. **Kotlin** — compile every `.kt` under `src\` to `build\kclasses`, targeting - JVM 1.8, against `android.jar` + coroutines + stdlib. -5. **Java** — compile every `.java` under `src\` to `build\jclasses` with - `--release 8`, against `android.jar` + the Kotlin output + stdlib. -6. **`jar`** both class trees, then **`d8`** them together with - `kotlin-stdlib`, `kotlin-stdlib-jdk7/8`, and `kotlinx-coroutines-core-jvm` - into `classes.dex` at `--min-api 29`. -7. **Package** — copy `resources.ap_` to the APK and add `classes*.dex` plus the - whole `lib\` tree using .NET's `ZipArchive` (Windows has no `zip` command). -8. **`zipalign -p -f 4`**, then **`apksigner sign`**, then - **`apksigner verify --verbose`**. - -### 8.4 Options +Options: ```powershell .\build_apk.ps1 -BuildCore # rebuild the Go .so first @@ -728,53 +871,35 @@ Pass `-NoDownload` to make a missing Kotlin a hard error instead. .\build_apk.ps1 -JavaHome 'C:\Program Files\Java\jdk-21.0.10' .\build_apk.ps1 -KotlinHome C:\kotlinc -NoDownload .\build_apk.ps1 -KotlinVersion 2.2.0 +.\build_apk.ps1 -RepoRoot C:\path\to\DragonTCP .\build_apk.ps1 -Keystore C:\keys\release.jks -KsPass … -KeyAlias … -KeyPass … ``` Go binaries alone: ```powershell -.\build_core.ps1 # android client .so + linux amd64/arm64 servers -.\build_core.ps1 -ClientOnly # just the .so +.\build_core.ps1 # android .so + linux client + linux amd64/arm64 servers +.\build_core.ps1 -ClientOnly # just the .so +.\build_core.ps1 -AndroidLibDir ``` -### 8.5 Windows-specific notes - -Three things differ from the shell build and are worth knowing before editing the -script: - -* **The Kotlin compiler is invoked as - `java -cp kotlin-compiler.jar org.jetbrains.kotlin.cli.jvm.K2JVMCompiler`, not - through `kotlinc.bat`.** `cmd.exe` treats `;` as an argument separator, so a - `-classpath a.jar;b.jar` handed to a batch file is split into two arguments and - the second jar is misread as a source file. Calling `java.exe` directly avoids - the batch tokenizer entirely. -* **`d8.bat` and `apksigner.bat` are still batch files.** Their arguments contain - no semicolons today, so they work — but a project path containing spaces or - semicolons could hit the same class of problem. -* **`zipalign -p -f 4` runs before signing.** The shell script omits it; it is - the canonical ordering and costs nothing. - -`android\.gitignore` keeps `build/`, `.tools/`, and the debug keystore out of -version control. - ---- - -## 9. Building on Linux and macOS +### 8.2 Linux and macOS ```bash -./build_core.sh # Go: android .so + linux amd64/arm64 servers +./build_core.sh ./android/build_apk.sh ./build_all.sh # both ``` `build_apk.sh` expects `ANDROID_SDK_ROOT` (or `ANDROID_HOME`) and a `KOTLIN_HOME` -pointing at a Kotlin distribution that bundles -`lib/kotlinx-coroutines-core-jvm.jar` — it defaults to -`~/.sdkman/candidates/kotlin/current`. Unlike the Windows script it does not -download anything for you. +pointing at a Kotlin distribution bundling `lib/kotlinx-coroutines-core-jvm.jar`; +it defaults to `~/.sdkman/candidates/kotlin/current`. Unlike the PowerShell script +it downloads nothing and does not build the Go core for you. -Go builds by hand, if you prefer: +`build_core.sh` does not build the Linux **client**; `build_core.ps1` does. Build +it by hand if you need it. + +Go builds by hand: ```bash cd core @@ -789,11 +914,53 @@ GOOS=linux GOARCH=amd64 CGO_ENABLED=0 \ -o ../bin/dragontcp-hybrid-server-linux-amd64 ./cmd/dragontcp-server ``` -CGO is off everywhere, so no NDK and no C toolchain are required for any target. +CGO is off for every target, so no NDK and no C toolchain are needed. + +### 8.3 What the build does + +1. **Resolve the toolchain and repo root**, and print what was chosen. +2. **Build the native core** if the `.so` is missing or `-BuildCore` was passed, + by calling `\build_core.ps1 -ClientOnly -AndroidLibDir \lib\arm64-v8a`. +3. **`aapt package`** — compile `res/`, pack `assets/`, bind the manifest against + `android.jar`, producing `resources.ap_`. +4. **Kotlin** — compile every `.kt` under `src\` to `build\kclasses`, JVM target + 1.8, against `android.jar` + coroutines + stdlib. +5. **Java** — compile every `.java` under `src\` to `build\jclasses` with + `--release 8`, against `android.jar` + the Kotlin output + stdlib. +6. **`jar`** both class trees, then **`d8`** them with `kotlin-stdlib`, + `kotlin-stdlib-jdk7/8`, and `kotlinx-coroutines-core-jvm` into `classes.dex` at + `--min-api 29`. +7. **Package** — copy `resources.ap_` to the APK and add `classes*.dex` plus the + whole `lib\` tree using .NET's `ZipArchive`. +8. **`zipalign -p -f 4`**, then **`apksigner sign`**, then + **`apksigner verify --verbose`**. + +### 8.4 Windows implementation notes + +* **The Kotlin compiler is invoked as + `java -cp kotlin-compiler.jar org.jetbrains.kotlin.cli.jvm.K2JVMCompiler`, not + through `kotlinc.bat`.** `cmd.exe` treats `;` as an argument separator, so + `-classpath a.jar;b.jar` handed to a batch file is split into two arguments and + the second jar is misread as a source file. Calling `java.exe` directly bypasses + the batch tokenizer. +* **`d8.bat` and `apksigner.bat` remain batch files.** Their arguments contain no + semicolons, but a project path containing spaces or semicolons could hit the + same class of problem. +* **Packaging uses .NET `ZipArchive`** because Windows has no `zip` command. +* **`zipalign` runs before signing**, which is the required ordering. +* **Native-tool stderr is not treated as failure.** The JVM emits a + `sun.misc.Unsafe` warning on recent JDKs and `apksigner` emits a native-access + warning; under `$ErrorActionPreference = 'Stop'` with a merged output stream + those would abort the build, so `Invoke-Tool` judges success by exit code alone. + +A `.gitignore` in the app directory keeps `build/`, `.tools/`, and the debug +keystore out of version control. --- -## 10. Running the server +## 9. Running + +### 9.1 Server ```bash sudo ./dragontcp-hybrid-server-linux-amd64 --port 53 --chunk-max 1048576 @@ -813,51 +980,11 @@ sudo ./dragontcp-hybrid-server-linux-amd64 \ --port 53 --chunk-max 1048576 --debug --debug-stats-interval 10s ``` -`sudo` is only needed because port 53 is privileged. If `systemd-resolved` or -`dnsmasq` already owns port 53, free it first or pick another port. The server -creates no TUN device and needs no NAT or `iptables` rules. +`sudo` is needed only because port 53 is privileged. If `systemd-resolved` or +`dnsmasq` already owns port 53, free it or choose another port. No TUN device, NAT, +or firewall rules are required. -| Flag | Default | Meaning | -|---|---|---| -| `--host` | `0.0.0.0` | Listen address | -| `--port` | `53` | Listen port | -| `--token` | empty | Optional shared secret | -| `--max-connections` | `20000` | Concurrent TCP connections | -| `--allow-private` | `false` | Allow private/loopback targets — **keep off in public** | -| `--dns-cache-ttl` | `30s` | Resolver cache lifetime | -| `--dns-cache-size` | `4096` | Cached hostnames | -| `--tcp-buffer` | `0` | Explicit socket buffers; 0 = OS autotuning | -| `--chunk-max` | `1048576` | Largest record the server accepts (32 B – 1 MiB) | -| `--chunk-buffered` | `256` | Per-session buffer in 64 KiB units (≈16 MiB) | -| `--chunk-poll-wait` | `200ms` | Long-poll wait for the first record of a batch | -| `--chunk-session-timeout` | `2m` | Idle session reaping | -| `--debug` | `false` | Session/connect/error logs plus periodic stats | -| `--debug-chunks` | `false` | Log every record — very verbose | -| `--debug-stats-interval` | `5s` | Statistics period; 0 disables | - ---- - -## 11. Running the client - -### On Android - -Install the APK, enter the server IP and port, grant the VPN prompt, connect. -Recommended starting point: - -```text -Server: YOUR_SERVER_IP -Port: 53 -Token: (match the server, or leave blank) -Max chunk: 1048576 -Min chunk: 32 -Concurrency: 1 -Reconnect every: 0 -Timeout (s): 2 -``` - -Use **OPEN LOGS** to watch the path probe and any adaptation. - -### As a CLI +### 9.2 Client CLI ```bash ./dragontcp-hybrid-client-linux-amd64 \ @@ -865,7 +992,58 @@ Use **OPEN LOGS** to watch the path probe and any adaptation. --listen-port 8080 --chunk-max 1048576 ``` -Then point anything at `http://127.0.0.1:8080` as an HTTP proxy. +Then point anything at `http://127.0.0.1:8080` as an HTTP proxy. The startup line +summarises the active configuration: + +```text +adaptive_chunk=true start=1048576 min=32 max=1048576 grow_after=16 pollers=1 \ +batch=5-5(pinned) reconnect_every=0 timeout=2s +``` + +### 9.3 Android + +Install the APK, enter the server address, grant the VPN prompt, connect. +A reasonable starting point: + +```text +Server: YOUR_SERVER_IP +Port: 53 +Token: (match the server, or leave blank) +Max chunk: 1048576 +Min chunk: 32 +Batch max: 1 +Batch min: 1 +Reconnect every: 0 +Timeout (s): 2 +``` + +Use **OPEN LOGS** to watch the path probe and any adaptation. + +--- + +## 10. Configuration reference + +### 10.1 Server flags + +| Flag | Default | Meaning | +|---|---|---| +| `--host` | `0.0.0.0` | Listen address | +| `--port` | `53` | Listen port | +| `--token` | empty | Optional shared secret | +| `--max-connections` | `20000` | Concurrent TCP connections | +| `--allow-private` | `false` | Allow private/loopback targets — keep off in public | +| `--dns-cache-ttl` | `30s` | Resolver cache lifetime | +| `--dns-cache-size` | `4096` | Cached hostnames | +| `--tcp-buffer` | `0` | Explicit socket buffers; 0 = OS autotuning | +| `--chunk-max` | `1048576` | Largest record accepted (32 B – 1 MiB) | +| `--chunk-buffered` | `256` | Per-session buffer in 64 KiB units (≈16 MiB) | +| `--chunk-poll-wait` | `200ms` | Long-poll wait for a batch's first record | +| `--chunk-session-timeout` | `2m` | Idle session reaping | +| `--debug` | `false` | Session/connect/error logs plus periodic stats | +| `--debug-chunks` | `false` | Log every record — very verbose | +| `--debug-stats-interval` | `5s` | Statistics period; 0 disables | + +### 10.2 Client flags | Flag | Default | Meaning | |---|---|---| @@ -875,150 +1053,149 @@ Then point anything at `http://127.0.0.1:8080` as an HTTP proxy. | `--transport` | `chunk` | Must be `chunk` | | `--max-connections` | `20000` | Concurrent proxied connections | | `--tcp-buffer` | `0` | Explicit socket buffers | -| `--chunk-start` | `1048576` | Initial record size (probe overrides it) | -| `--chunk-min` | `32` | Floor | -| `--chunk-max` | `1048576` | Ceiling, further clamped by the server | -| `--chunk-adaptive` | `true` | Enable runtime resizing | +| `--chunk-start` | `1048576` | Initial record size (the probe overrides it) | +| `--chunk-min` | `32` | Record size floor | +| `--chunk-max` | `1048576` | Record size ceiling, further clamped by the server | +| `--chunk-adaptive` | `true` | Enable runtime record resizing | | `--chunk-grow-after` | `16` | Successes before growing | -| `--chunk-adapt-log` | `true` | Print size changes | +| `--chunk-adapt-log` | `true` | Print size and batch changes | | `--chunk-size` | `0` | Legacy: pins start/min/max and disables adaptation | | `--chunk-concurrency` | `1` | Download batch ceiling, 1–256 | +| `--chunk-concurrency-min` | `1` | Download batch floor, 1–256; equal to the ceiling pins the depth | | `--chunk-reconnect-every` | `0` | 0 persistent, 1 auto, N rotate | | `--chunk-poll-delay` | `2ms` | Pause after an empty poll | | `--chunk-timeout` | `2s` | Per-record transaction timeout | -| `--chunk-pollers` | `1` | Reserved compatibility knob; unused | +| `--chunk-pollers` | `1` | Reserved compatibility knob; accepted but unused | + +The `concurrency` flag names are historical. They control the download **batch +depth** described in §5.6, not any form of threading. + +### 10.3 Fixed timings + +| Where | Value | What | +|---|---|---| +| Client | 15 s | Local-connection handshake deadline | +| Client | 10 s | Dial timeout to the server | +| Client | 2.5 s | Per-probe timeout cap | +| Client | 20 s | Whole-probe timeout per direction | +| Client | 30 min | Path profile cache lifetime | +| Client | 30 ms | Pause between failed record retries | +| Server | 30 s | Per-request read deadline | +| Server | 2 ms | Small-read coalescing window | +| Server | 30 s | Session sweep interval | +| Android | 10 s | Wait for the local proxy to accept | +| Android | 1200 + 800 ms | Graceful then forced core shutdown | +| Android | 400 ms | TCP retransmit timeout | +| Android | 5 s | Session housekeeping interval | --- -## 12. Tuning guide +## 11. Tuning -**Start with the defaults.** Path probing already picks sensible sizes; most +Start with the defaults. Path probing already picks sensible sizes, and most manual tuning makes things worse. -* **Throughput feels capped.** Raise `Concurrency` to 4–16. More records per - request is the main lever when latency to the server is high, because each - round trip returns more data. Watch the logs — if you see repeated - `adaptive download pipeline: N -> N/2`, the path cannot sustain that depth. +* **Throughput feels capped.** Raise `Batch max` to 4–16 and leave `Batch min` at + 1. More records per request is the main lever when latency to the server is + high, because each round trip returns more data. If the log repeatedly shows + `adaptive download batch: N -> N/2`, the path cannot sustain that depth. +* **The batch keeps collapsing to 1 and throughput dies with it.** Some paths only + deliver correctly at one specific number of records. Try `Batch max = Batch min + = N` for a few values of N and leave it pinned at whichever works. Pinned mode + never halves and is never trimmed by the 1 MiB batch cap. +* **You want a floor but still want headroom.** Set `Batch min` to the smallest + depth the path tolerates and `Batch max` higher; adaptation then works inside + that window instead of falling all the way to 1. * **Frequent `after transport failure` lines.** The network is dropping large - records. Lower `Max chunk` to something the probe already found safe (1400 is - common) so the client stops rediscovering the limit. -* **Connection dies after a while, then recovers.** A middlebox is capping - requests per connection. Set `Reconnect every` to something like 8–32. -* **Nothing connects at all, but the probe succeeds.** Check the token matches, - and check that the server is not refusing the target because it resolves to a - private address. -* **High latency, low bandwidth link.** Leave `--tcp-buffer` at 0 first. Only if - you have a small number of high-BDP connections is `1048576` or `4194304` - worth trying; on many connections it costs memory for nothing. -* **Server memory.** Each session can hold up to `--chunk-buffered × 64 KiB` - (default 16 MiB). With many concurrent sessions, lower it. + records. Lower `Max chunk` to a size the probe already found safe — 1400 is + common — so the client stops rediscovering the limit. +* **Connection dies periodically, then recovers.** A middlebox is capping requests + per connection. Set `Reconnect every` to 8–32. +* **High-latency, high-bandwidth link.** Leave `--tcp-buffer` at 0 first. Only for + a small number of high-BDP connections is `1048576` or `4194304` worth trying; + across many connections it costs memory for nothing. +* **Server memory.** Each session can hold `--chunk-buffered × 64 KiB` (default + 16 MiB). Lower it when running many concurrent sessions. --- -## 13. Troubleshooting +## 12. Troubleshooting -### Build +### 12.1 Build | Symptom | Cause and fix | |---|---| -| `Android SDK not found` | Set `ANDROID_SDK_ROOT` or pass `-SdkRoot 'D:\Android\Sdk'`. | -| `No usable build-tools found` | Install build-tools via the SDK Manager; the script needs `aapt`, `d8`, `apksigner`, `zipalign` together. | +| `Android SDK not found` | Set `ANDROID_SDK_ROOT` or pass `-SdkRoot`. | +| `No usable build-tools found` | Install build-tools; `aapt`, `d8`, `apksigner`, and `zipalign` must all be present in one version. | | `Missing …\jar.exe (a JRE is not enough)` | You have a JRE or the `javapath` shim. Install a JDK and set `JAVA_HOME`. | -| `Kotlin download failed` | No network, or a proxy. Download `kotlin-compiler-.zip` by hand, extract it, and pass `-KotlinHome \kotlinc`. | -| `source entry is not a Kotlin file: …jar` | You reintroduced `kotlinc.bat`. See §8.5 — call the compiler jar through `java.exe`. | +| `Kotlin download failed` | No network, or a proxy. Download `kotlin-compiler-.zip` manually, extract it, pass `-KotlinHome \kotlinc`. | +| `source entry is not a Kotlin file: …jar` | The compiler is being called through `kotlinc.bat`. See §8.4. | | `libdragontcp_client.so is missing` | Install Go and run `.\build_core.ps1 -ClientOnly`, or pass `-BuildCore`. | -| `run ..\build_core.ps1 first` on Linux | Use `./build_core.sh`; the shell script does not build the core for you. | +| `…but no core\go.mod was found above` | Run the script from the app directory, or pass `-RepoRoot`. | -### Runtime +### 12.2 Runtime | Symptom | Cause and fix | |---|---| | `CONNECT failed: Server is required` | Empty server field. | -| `Local proxy did not start` | The Go core died within 10 s. Open the logs; usually a bad flag or an unusable port. | -| `DragonTCP core exited: N` | The core process died while connected. The whole tunnel is torn down deliberately. | +| `Local proxy did not start` | The core died within 10 s. Check the logs; usually a bad flag or an unusable port. | +| `DragonTCP core exited: N` | The core died while connected; the tunnel is torn down deliberately. | | `authentication failed` | Token mismatch between app and server. | -| `target resolves only to blocked addresses` | The destination is private/loopback. Intentional; `--allow-private` on the server overrides it, at real risk. | -| `download offset N was already acknowledged` | Client and server disagree on stream position — almost always a stale session after a restart. Reconnect. | -| `upload gap: got N expected M` | Same, in the upload direction. Reconnect. | -| DNS works, QUIC/UDP apps do not | By design: only DNS is carried over UDP. Apps fall back to TCP. | -| No IPv6 anywhere | By design: IPv6 is captured and blackholed to prevent bypass. | +| `target resolves only to blocked addresses` | The destination is private or special-use. Intentional; see §6.5. | +| `download offset N was already acknowledged` | Client and server disagree on stream position, almost always a stale session after a restart. Reconnect. | +| `upload gap: got N expected M` | The same, in the upload direction. Reconnect. | +| `server maximum chunk N is below client minimum M` | Raise the server's `--chunk-max` or lower the client's `Min chunk`. | +| `download failed at minimum chunk` | Eight consecutive failures at the smallest record size — the path is not passing traffic at all. | +| DNS works, QUIC/UDP apps do not | By design: only DNS is carried over UDP (§7.7). | +| No IPv6 anywhere | By design: IPv6 is captured and blackholed (§7.4). | +| Connections hang instead of failing fast | Expected only for non-SYN packets to unknown flows, which receive an RST. Anything else warrants the logs. | --- -## 14. Version history +## 13. Testing -**Hybrid v1** — the current wire protocol. +```bash +cd core && go test ./... +``` -* Kept the lightweight Android TUN → local HTTP proxy architecture. -* Replaced ASCII `UP`/`OK`/`CPUSH`/`CPULL` framing with compact binary records. -* Replaced the fixed `0xAD` XOR with a changing SHA-256-derived keystream. -* Added automatic upload/download path-size probing with binary search. -* Kept separate adaptive sizes per direction. -* Added download batching with adaptive pipeline depth. -* Records up to 1 MiB; token optional; TCP/53 default. -* Reconnect selectable: persistent, auto, or forced rotation. - -**Hybrid v2** — withdrawn. Introduced a multi-request upload pipeline and -65 535-record mega-batches; both proved unreliable. None of it is present here. - -**Hybrid v3 "SafeSpeed"** — built directly from the confirmed-working v1. - -* Wire encoding byte-for-byte unchanged from v1: same headers, same 16-byte - session IDs, same keystream, same probes, same upload transactions, same - server framing. -* Download concurrency became user-configurable, 1–256, default 1. -* `1` keeps the pipeline fixed at one. Above 1, depth starts at the ceiling, - halves on transport failure, and grows by one on success, staying in `1..N`. -* Server cap remains 256 records per batch. -* Upload remains one framed request followed by one response ACK. - ---- - -## 15. Testing and validation - -`cd core && go test ./...` covers: +Coverage: * the masking round-trip, and that different sequences produce different wire bytes, -* the adaptive sizer recovering from the minimum rather than latching there, -* `reconnectEvery == 0` meaning persistent. +* the adaptive record sizer recovering from the minimum rather than latching + there, +* `reconnectEvery == 0` meaning persistent, +* a pinned batch never growing or shrinking across repeated failures, +* a pinned batch surviving the ~1 MiB per-batch cap while an unpinned one is still + trimmed by it, +* an adaptive batch halving down to its floor and growing back to its ceiling, +* `1..1` being treated as pinned. -Current status on this checkout: +Beyond unit tests, the transport has been exercised with an 8 MiB HTTP download +verified by SHA-256, an HTTPS `CONNECT` download verified byte for byte, +persistent and forced-rotation connection modes, and a server restricted to +1400-byte records where probing selected 1400 automatically and the download still +completed correctly. -```text -ok dragontcp/cmd/dragontcp-client -ok dragontcp/cmd/dragontcp-server -? dragontcp/internal/protocol [no test files] -ok dragontcp/internal/wire -``` - -Beyond unit tests, the transport was exercised with: - -* an 8 MiB HTTP download through the proxy, verified by SHA-256, -* an HTTPS `CONNECT` download verified byte for byte, -* persistent connection mode, -* forced `reconnect-every-1` mode, -* a server restricted to 1400-byte records, where probing selected 1400 - automatically and the download still completed correctly. - -`SHA256SUMS` records digests for the published binaries. +`SHA256SUMS` records digests for the built binaries; the digest for +`android/lib/arm64-v8a/libdragontcp_client.so` matches the copy packaged inside +the APK. --- -## 16. Licensing +## 14. Licensing The Android userspace TCP/IP stack under `android/src/tech/xvanturing/freeproxy/` is adapted from **FreeProxy** by xVanTuring, licensed under the **Apache License 2.0**. Modified files carry a marker comment at the top. -* Full attribution: [`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md) -* Full license text: [`licenses/FreeProxy-APACHE-2.0.txt`](licenses/FreeProxy-APACHE-2.0.txt) -* Both are also shipped inside the APK under `assets/`. +* Attribution: [`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md) +* License text: [`licenses/FreeProxy-APACHE-2.0.txt`](licenses/FreeProxy-APACHE-2.0.txt) -Those notice files are a license condition, not documentation — they are kept -separate from this README deliberately, and should not be folded into it or -deleted. +Both are also shipped inside the APK under `assets/`. They are a license +condition rather than documentation, and must not be folded into this file or +removed. The remaining DragonTCP glue, UI, Go transport, and server code is provided as part of this project. diff --git a/SHA256SUMS b/SHA256SUMS index 1dd9250..a9ba0ab 100644 --- a/SHA256SUMS +++ b/SHA256SUMS @@ -1,4 +1,4 @@ -bff5e7d9bd9b133ceb90987ad5914f1f26d085ad699cd7d0fc7341e9f713c52f bin/dragontcp-hybrid-server-linux-amd64 -333718a76ce4a89d968cd7fb4aa74f0365a861a3f0ac36fe933085ca5d67fa11 bin/dragontcp-hybrid-server-linux-arm64 -94f3fb7cf895b79f0db355ac51b8f11cd14a7d8641e552dfb1b31aad326666ff bin/dragontcp-hybrid-client-linux-amd64 -ed361cf7a6d72b1c22a111875957e5111f3088bc2843fff02361eb2bc2dc967b android/lib/arm64-v8a/libdragontcp_client.so +4e4a16f6c537f2c6be5c104f6e50ea907122092a21402a70506fe12e6f144bb7 bin/dragontcp-hybrid-server-linux-amd64 +39a6bb16cbddefb50e791220747f5cd01b0c894992c657b24c6c2e5d1501299f bin/dragontcp-hybrid-server-linux-arm64 +66ad3741d84b4e73098912725af762824bec629adab5522a5e161b4e23b79dee bin/dragontcp-hybrid-client-linux-amd64 +fa6b41cc8c2999932cc78f731cb249cab677af9df7701c491fb688f9cbcb089d android/lib/arm64-v8a/libdragontcp_client.so diff --git a/android/.gitignore b/android/.gitignore new file mode 100644 index 0000000..e43ee2f --- /dev/null +++ b/android/.gitignore @@ -0,0 +1,4 @@ +# Build outputs and the auto-downloaded Kotlin compiler (~85 MB). +/build/ +/.tools/ +/dragontcp-lite-debug.jks diff --git a/android/android/AndroidManifest.xml b/android/AndroidManifest.xml similarity index 95% rename from android/android/AndroidManifest.xml rename to android/AndroidManifest.xml index 06095da..07161df 100644 --- a/android/android/AndroidManifest.xml +++ b/android/AndroidManifest.xml @@ -1,8 +1,8 @@ + android:versionCode="13" + android:versionName="13.0-hybrid"> diff --git a/android/README.md b/android/README.md deleted file mode 100644 index 11a32de..0000000 --- a/android/README.md +++ /dev/null @@ -1,239 +0,0 @@ -# DragonTCP Hybrid Transport - -This branch keeps the lightweight DragonTCP Android architecture that was -working well: - -```text -Android apps - -> Android VpnService userspace TCP adapter - -> local HTTP/HTTPS CONNECT proxy on 127.0.0.1:8080 - -> DragonTCP transport over TCP/53 - -> DragonTCP server - -> destination website -``` - -The heavy raw-IP DragonTCP VPN server is **not** used. The Linux server is -again only a relay/proxy server; it does not create a TUN interface and does -not need NAT/iptables. - -## What changed on the wire - -The old DragonTCP transport used a fixed text/frame pattern: - -```text -UP + request id + length + XOR(0xAD, ASCII CPUSH/CPULL/...) -OK + request id + length + XOR(0xAD, response) -``` - -The Hybrid transport replaces that network layer with compact binary records: - -```text -Request header (29 bytes) - 1 byte mode - 16 bytes random session ID - 8 bytes sequence / stream offset - 4 bytes payload length - -Response header (5 bytes) - 1 byte status - 4 bytes body length -``` - -There are no constant `UP`, `OK`, `CPUSH`, `CPULL`, or `DATA` strings on the -wire. - -Payload bytes are masked with a sequence-dependent SHA-256 counter keystream. -The operation is still XOR-based, but the XOR bytes change with session, -mode, sequence, direction, and block number instead of using the same 0xAD -byte for every position. - -This is traffic shaping/obfuscation, not authenticated encryption. - -## Path probing - -Before the first proxied connection, the client probes the phone-to-server -path once and caches the result for 30 minutes. - -Upload and download are tested independently against a ladder from small -records through 1 MiB. A binary search is used, so it does not need to fail at -every size between the maximum and minimum. - -Example log: - -```text -path probe: upload=32768 download=1400 persistent=true -``` - -If the server is capped at 1400 bytes, the client can discover: - -```text -path probe: upload=1400 download=1400 persistent=true -``` - -instead of beginning at 1 MiB and repeatedly halving during real traffic. - -## Separate upload/download sizing - -Upload and download maintain independent record sizes. Runtime adaptation is -still present as a safety net if network conditions change after the initial -probe. - -Useful log events remain concise: - -```text -adaptive upload chunk: 32768 -> 16384 after transport failure -adaptive download pipeline: 64 -> 32 after transport failure -adaptive download chunk: 1400 -> 700 after transport failure -``` - -## Batched downloads - -One download request can receive many ordered DATA responses. The client uses -one download worker but pipelines up to 256 response records per request, -subject to an approximately 1 MiB useful-data cap per batch. - -This is especially important on restricted paths. If the safe record size is -32 bytes, one TCP/53 request can still bring back many 32-byte records instead -of requiring a new request for every 32 useful bytes. - -## Reconnect behavior - -The transport supports persistent TCP connections and forced connection -rotation. - -Core/CLI semantics: - -```text ---chunk-reconnect-every 0 persistent ---chunk-reconnect-every 1 automatic compatibility mode ---chunk-reconnect-every N rotate after N logical requests, N >= 2 -``` - -Automatic mode probes persistent request reuse. If it works, DragonTCP keeps -the connection. If it does not, DragonTCP falls back to one logical request -per TCP connection. - -The supplied APK is based on the current Logs Page UI, whose Reconnect field -still requires a value of at least 1. In this APK, leave it at `1` for Auto. -The full Android source in this package also contains the newer explicit -`0 = persistent` UI behavior for rebuilding with the Android SDK. - -## Optional token - -The token remains optional. - -Server without token: - -```bash -sudo ./dragontcp-hybrid-server-linux-amd64 --chunk-max 1048576 -``` - -Server with token: - -```bash -sudo ./dragontcp-hybrid-server-linux-amd64 \ - --token 'YOUR_SECRET' \ - --chunk-max 1048576 -``` - -## Server - -Default server port is TCP/53: - -```bash -sudo ./dragontcp-hybrid-server-linux-amd64 \ - --port 53 \ - --chunk-max 1048576 -``` - -Useful debug mode: - -```bash -sudo ./dragontcp-hybrid-server-linux-amd64 \ - --port 53 \ - --chunk-max 1048576 \ - --debug \ - --debug-stats-interval 10s -``` - -The server is a lightweight TCP relay. It does not create Linux TUN devices. - -## Android - -Install `DragonTCP-Hybrid-Android-arm64.apk`, configure the server IP/port, -and connect as before. - -Recommended initial settings: - -```text -Server: YOUR_SERVER_IP -Port: 53 -Token: optional -Max chunk: 1048576 -Min chunk: 32 -Reconnect: 1 (Auto in supplied APK) -Timeout: 2 -``` - -The Android TUN frontend continues to point applications at the local -DragonTCP HTTP proxy. DNS continues to use the existing lightweight adapter's -DNS-over-proxy path. - -## Source layout - -```text -core/ - cmd/dragontcp-client/ - cmd/dragontcp-server/ - internal/protocol/ # TCP tuning + legacy helpers - internal/wire/ # new compact binary framing + changing mask - -android/ - src/com/dragontcp/client/ - src/tech/xvanturing/freeproxy/ - lib/arm64-v8a/libdragontcp_client.so -``` - -## Building the Go core - -```bash -cd core - -go test ./... - -go build -trimpath -ldflags='-s -w' \ - -o ../bin/dragontcp-hybrid-server-linux-amd64 \ - ./cmd/dragontcp-server - -GOOS=android GOARCH=arm64 CGO_ENABLED=0 \ - go build -trimpath -ldflags='-s -w' \ - -o ../android/lib/arm64-v8a/libdragontcp_client.so \ - ./cmd/dragontcp-client -``` - -`android/build_apk.sh` contains the source build procedure for the Android -application when an Android SDK and Kotlin compiler are installed. - -## Validation performed - -The new Go transport was tested with: - -- Go unit tests for client/server/wire code. -- changing-mask round-trip test. -- 8 MiB HTTP download through the DragonTCP proxy with SHA-256 equality. -- HTTPS CONNECT download through DragonTCP with byte-for-byte equality. -- persistent connection mode. -- forced reconnect-every-1 mode. -- a server restricted to 1400-byte records, where path probing selected 1400 - bytes automatically and the download still completed correctly. - -The supplied APK embeds the exact Android ARM64 Go core built from this source. - - -## SafeSpeed v3 - -This release is built directly from the confirmed-working Hybrid v1. The wire protocol is unchanged: same compact binary request/response headers, same 16-byte session IDs, same SHA-256-derived XOR keystream, same probe packets, same upload transaction behavior, and same server framing. - -The only transport scheduling change is that the download pipeline starts at 256 records (the same maximum already supported by v1) instead of starting at 32 and slowly growing one record per successful request. This is especially important when the probed record size is small. - -The broken Hybrid v2 changes are intentionally absent: no multi-request upload pipeline and no 65,535-record mega-batches. diff --git a/android/RELEASE_NOTES.md b/android/RELEASE_NOTES.md deleted file mode 100644 index 55af16a..0000000 --- a/android/RELEASE_NOTES.md +++ /dev/null @@ -1,12 +0,0 @@ -# DragonTCP Hybrid v1 - -- Keeps the lightweight Android TUN -> local HTTP proxy architecture. -- Replaces ASCII UP/OK/CPUSH/CPULL framing with compact binary records. -- Replaces fixed XOR 0xAD payload masking with a changing SHA-256-derived XOR stream. -- Adds automatic upload/download path-size probing. -- Keeps separate upload and download adaptive sizes. -- Adds download batching and adaptive pipeline depth. -- Supports chunks up to 1 MiB. -- Token remains optional. -- TCP/53 remains the default server transport. -- Reconnect is optional: persistent, Auto, or forced rotation. diff --git a/android/RELEASE_NOTES_SAFE_SPEED.md b/android/RELEASE_NOTES_SAFE_SPEED.md deleted file mode 100644 index 582c434..0000000 --- a/android/RELEASE_NOTES_SAFE_SPEED.md +++ /dev/null @@ -1,7 +0,0 @@ -# DragonTCP Hybrid v3 SafeSpeed - -- Exact Hybrid v1 wire encoding retained. -- Download pipeline starts at 256 records. -- Server cap remains the v1 value of 256 records. -- Upload remains one framed request followed by one response ACK. -- No v2 mega-batch behavior. diff --git a/android/SHA256SUMS b/android/SHA256SUMS deleted file mode 100644 index 4e759ff..0000000 --- a/android/SHA256SUMS +++ /dev/null @@ -1,4 +0,0 @@ -bff5e7d9bd9b133ceb90987ad5914f1f26d085ad699cd7d0fc7341e9f713c52f bin/dragontcp-hybrid-server-linux-amd64 -333718a76ce4a89d968cd7fb4aa74f0365a861a3f0ac36fe933085ca5d67fa11 bin/dragontcp-hybrid-server-linux-arm64 -94f3fb7cf895b79f0db355ac51b8f11cd14a7d8641e552dfb1b31aad326666ff bin/dragontcp-hybrid-client-linux-amd64 -af450a117e302eba065feb3a8e4f9b3e0709b3456e821812cb2dacc150041894 android/lib/arm64-v8a/libdragontcp_client.so diff --git a/android/android/assets/THIRD_PARTY_NOTICES.md b/android/android/assets/THIRD_PARTY_NOTICES.md deleted file mode 100644 index c8e6795..0000000 --- a/android/android/assets/THIRD_PARTY_NOTICES.md +++ /dev/null @@ -1,20 +0,0 @@ -# Third-party notices - -DragonTCP Lite VPN includes portions of the Android userspace TCP/IP stack from -**FreeProxy** by xVanTuring. The upstream project is licensed under the Apache -License, Version 2.0. - -Included/adapted upstream areas: - -- `android/src/tech/xvanturing/freeproxy/vpn/TunnelEngine.kt` -- `android/src/tech/xvanturing/freeproxy/vpn/TcpSession.kt` -- `android/src/tech/xvanturing/freeproxy/vpn/UdpSession.kt` -- `android/src/tech/xvanturing/freeproxy/vpn/TunWriter.kt` -- `android/src/tech/xvanturing/freeproxy/vpn/net/*` -- the `SocketProtector` interface - -DragonTCP-specific modifications are marked in modified source files. The full -Apache 2.0 license is included at `licenses/FreeProxy-APACHE-2.0.txt`. - -The rest of the DragonTCP-specific glue, UI, Go transport, and server code in -this bundle is provided as part of this generated project. diff --git a/android/android/lib/arm64-v8a/libdragontcp_client.so b/android/android/lib/arm64-v8a/libdragontcp_client.so deleted file mode 100644 index 7a219eb..0000000 Binary files a/android/android/lib/arm64-v8a/libdragontcp_client.so and /dev/null differ diff --git a/android/android/assets/FreeProxy-APACHE-2.0.txt b/android/assets/FreeProxy-APACHE-2.0.txt similarity index 100% rename from android/android/assets/FreeProxy-APACHE-2.0.txt rename to android/assets/FreeProxy-APACHE-2.0.txt diff --git a/android/THIRD_PARTY_NOTICES.md b/android/assets/THIRD_PARTY_NOTICES.md similarity index 100% rename from android/THIRD_PARTY_NOTICES.md rename to android/assets/THIRD_PARTY_NOTICES.md diff --git a/android/bin/dragontcp-hybrid-client-linux-amd64 b/android/bin/dragontcp-hybrid-client-linux-amd64 deleted file mode 100644 index cee6698..0000000 Binary files a/android/bin/dragontcp-hybrid-client-linux-amd64 and /dev/null differ diff --git a/android/bin/dragontcp-hybrid-server-linux-amd64 b/android/bin/dragontcp-hybrid-server-linux-amd64 deleted file mode 100644 index 59abf36..0000000 Binary files a/android/bin/dragontcp-hybrid-server-linux-amd64 and /dev/null differ diff --git a/android/bin/dragontcp-hybrid-server-linux-arm64 b/android/bin/dragontcp-hybrid-server-linux-arm64 deleted file mode 100644 index 1bccd9b..0000000 Binary files a/android/bin/dragontcp-hybrid-server-linux-arm64 and /dev/null differ diff --git a/android/bin/dragontcp-hybrid-v3-safespeed-client-linux-amd64 b/android/bin/dragontcp-hybrid-v3-safespeed-client-linux-amd64 deleted file mode 100644 index a3a1811..0000000 Binary files a/android/bin/dragontcp-hybrid-v3-safespeed-client-linux-amd64 and /dev/null differ diff --git a/android/bin/dragontcp-hybrid-v3-safespeed-server-linux-amd64 b/android/bin/dragontcp-hybrid-v3-safespeed-server-linux-amd64 deleted file mode 100644 index 59abf36..0000000 Binary files a/android/bin/dragontcp-hybrid-v3-safespeed-server-linux-amd64 and /dev/null differ diff --git a/android/bin/dragontcp-hybrid-v3-safespeed-server-linux-arm64 b/android/bin/dragontcp-hybrid-v3-safespeed-server-linux-arm64 deleted file mode 100644 index 1bccd9b..0000000 Binary files a/android/bin/dragontcp-hybrid-v3-safespeed-server-linux-arm64 and /dev/null differ diff --git a/android/bin/dragontcp-lite-client-linux-amd64 b/android/bin/dragontcp-lite-client-linux-amd64 deleted file mode 100644 index a7b3b7e..0000000 Binary files a/android/bin/dragontcp-lite-client-linux-amd64 and /dev/null differ diff --git a/android/bin/dragontcp-lite-server-linux-amd64 b/android/bin/dragontcp-lite-server-linux-amd64 deleted file mode 100644 index fe828f4..0000000 Binary files a/android/bin/dragontcp-lite-server-linux-amd64 and /dev/null differ diff --git a/android/bin/dragontcp-lite-server-linux-arm64 b/android/bin/dragontcp-lite-server-linux-arm64 deleted file mode 100644 index 1595756..0000000 Binary files a/android/bin/dragontcp-lite-server-linux-arm64 and /dev/null differ diff --git a/android/build_all.sh b/android/build_all.sh deleted file mode 100644 index 29013ca..0000000 --- a/android/build_all.sh +++ /dev/null @@ -1,5 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail -ROOT="$(cd "$(dirname "$0")" && pwd)" -"$ROOT/build_core.sh" -"$ROOT/android/build_apk.sh" diff --git a/android/build_apk.cmd b/android/build_apk.cmd new file mode 100644 index 0000000..687f973 --- /dev/null +++ b/android/build_apk.cmd @@ -0,0 +1,9 @@ +@echo off +REM Double-clickable wrapper around build_apk.ps1. +REM Any arguments are forwarded, e.g. build_apk.cmd -BuildCore +setlocal +powershell.exe -NoProfile -ExecutionPolicy Bypass -File "%~dp0build_apk.ps1" %* +set RC=%ERRORLEVEL% +if not "%RC%"=="0" echo.& echo BUILD FAILED (exit %RC%) +if "%~1"=="" pause +exit /b %RC% diff --git a/android/build_apk.ps1 b/android/build_apk.ps1 new file mode 100644 index 0000000..83423b3 --- /dev/null +++ b/android/build_apk.ps1 @@ -0,0 +1,337 @@ +<# +.SYNOPSIS + Windows build for the DragonTCP Lite APK (no Gradle, no Android Studio). + +.DESCRIPTION + Port of build_apk.sh. Compiles the Kotlin TUN adapter and the Java UI/service, + dexes them with d8, packages resources with aapt, aligns with zipalign and + signs with apksigner. + + Run it from the directory that holds AndroidManifest.xml. The Go core lives + in /core; the repo root is located by walking up from this script until + a core\go.mod is found, so the script keeps working if the tree is nested. + + Everything else is auto-detected where possible: + * Android SDK - ANDROID_SDK_ROOT / ANDROID_HOME, or -SdkRoot + * build-tools - newest installed version that has aapt/d8/apksigner/zipalign + * platform - android-35 if present, else newest installed + * JDK - JAVA_HOME, then javac on PATH, then C:\Program Files\Java\* + * Kotlin - KOTLIN_HOME, or downloaded once into .\.tools\kotlinc- + * native lib - built via \build_core.ps1 if the .so is missing + +.EXAMPLE + .\build_apk.ps1 + .\build_apk.ps1 -BuildCore # rebuild the Go .so first + .\build_apk.ps1 -BuildTools 35.0.0 -Platform android-35 + .\build_apk.ps1 -Keystore C:\keys\rel.jks -KsPass secret -KeyAlias rel -KeyPass secret +#> +[CmdletBinding()] +param( + [string]$SdkRoot, + [string]$BuildTools, + [string]$Platform, + [string]$JavaHome, + [string]$KotlinHome, + [string]$KotlinVersion = '2.1.21', + [string]$RepoRoot, + [string]$Keystore, + [string]$KsPass = 'dragontcp', + [string]$KeyAlias = 'dragontcp', + [string]$KeyPass = 'dragontcp', + # Rebuild the Go native library before packaging. + [switch]$BuildCore, + # Fail instead of downloading the Kotlin compiler. + [switch]$NoDownload +) + +$ErrorActionPreference = 'Stop' +$Root = $PSScriptRoot +$ProgressPreference = 'SilentlyContinue' # Invoke-WebRequest is ~10x faster without it + +function Step { param([string]$m) Write-Host "[apk] $m" -ForegroundColor Cyan } +function Die { param([string]$m) throw $m } + +function Invoke-Tool { + param([string]$Exe, [string[]]$ToolArgs, [string]$What) + # These tools write harmless warnings to stderr (the JVM's sun.misc.Unsafe + # notice, apksigner's native-access notice). Under $ErrorActionPreference = + # 'Stop' with a merged stream those become terminating errors, so judge + # success by the exit code alone. + $saved = $ErrorActionPreference + $ErrorActionPreference = 'Continue' + try { & $Exe @ToolArgs } finally { $ErrorActionPreference = $saved } + if ($LASTEXITCODE -ne 0) { Die "$What failed (exit $LASTEXITCODE): $Exe" } +} + +if (-not (Test-Path (Join-Path $Root 'AndroidManifest.xml'))) { + Die "No AndroidManifest.xml next to this script. Run it from the Android app directory." +} + +# ------------------------------------------------------------------ repo root +if (-not $RepoRoot) { + # Walk up collecting every ancestor that looks like a checkout. A tree can + # contain more than one copy of core\, so prefer the one that also carries + # build_core.ps1 and only fall back to the nearest bare core\go.mod. + $fallback = $null + $probe = $Root + for ($i = 0; $i -lt 8 -and $probe; $i++) { + if (Test-Path (Join-Path $probe 'core\go.mod')) { + if (Test-Path (Join-Path $probe 'build_core.ps1')) { $RepoRoot = $probe; break } + if (-not $fallback) { $fallback = $probe } + } + $parent = Split-Path $probe -Parent + if ($parent -eq $probe) { break } + $probe = $parent + } + if (-not $RepoRoot) { $RepoRoot = $fallback } +} +if ($RepoRoot) { $RepoRoot = (Resolve-Path $RepoRoot).Path } + +# ---------------------------------------------------------------- Android SDK +if (-not $SdkRoot) { + foreach ($c in $env:ANDROID_SDK_ROOT, $env:ANDROID_HOME, + "$env:LOCALAPPDATA\Android\Sdk", 'C:\Android\Sdk') { + if ($c -and (Test-Path $c)) { $SdkRoot = $c; break } + } +} +if (-not $SdkRoot -or -not (Test-Path $SdkRoot)) { + Die "Android SDK not found. Set ANDROID_SDK_ROOT (or pass -SdkRoot 'D:\Android\Sdk')." +} +$SdkRoot = (Resolve-Path $SdkRoot).Path + +$needTools = @{ aapt = 'aapt.exe'; d8 = 'd8.bat'; apksigner = 'apksigner.bat'; zipalign = 'zipalign.exe' } + +function Test-BuildTools { + param([string]$Dir) + foreach ($f in $needTools.Values) { if (-not (Test-Path (Join-Path $Dir $f))) { return $false } } + return $true +} + +$btRoot = Join-Path $SdkRoot 'build-tools' +if (-not (Test-Path $btRoot)) { Die "No build-tools under $SdkRoot. Install them via the SDK Manager." } +if ($BuildTools) { + $BT = Join-Path $btRoot $BuildTools + if (-not (Test-BuildTools $BT)) { Die "build-tools $BuildTools is missing one of: $($needTools.Values -join ', ')" } +} else { + $candidates = Get-ChildItem $btRoot -Directory | Sort-Object { + try { [version]$_.Name } catch { [version]'0.0.0' } + } -Descending + $BT = $null + foreach ($c in $candidates) { if (Test-BuildTools $c.FullName) { $BT = $c.FullName; $BuildTools = $c.Name; break } } + if (-not $BT) { Die "No usable build-tools found under $btRoot (need $($needTools.Values -join ', '))." } +} + +$platRoot = Join-Path $SdkRoot 'platforms' +if ($Platform) { + $AJ = Join-Path $platRoot "$Platform\android.jar" +} else { + $preferred = Join-Path $platRoot 'android-35\android.jar' + if (Test-Path $preferred) { + $Platform = 'android-35'; $AJ = $preferred + } else { + $p = Get-ChildItem $platRoot -Directory -ErrorAction SilentlyContinue | + Where-Object { Test-Path (Join-Path $_.FullName 'android.jar') } | + Sort-Object { [int]($_.Name -replace '\D', '') } -Descending | + Select-Object -First 1 + if (-not $p) { Die "No android.jar found under $platRoot. Install a platform via the SDK Manager." } + $Platform = $p.Name; $AJ = Join-Path $p.FullName 'android.jar' + } +} +if (-not (Test-Path $AJ)) { Die "Android platform jar not found: $AJ" } + +# ----------------------------------------------------------------------- JDK +if (-not $JavaHome) { + if ($env:JAVA_HOME -and (Test-Path "$env:JAVA_HOME\bin\javac.exe")) { + $JavaHome = $env:JAVA_HOME + } else { + $jc = Get-Command javac.exe -ErrorAction SilentlyContinue + # javapath\javac.exe is a shim: it can compile but has no jar/keytool next to it. + if ($jc -and (Test-Path (Join-Path (Split-Path $jc.Source) 'jar.exe'))) { + $JavaHome = Split-Path (Split-Path $jc.Source) + } else { + $j = Get-ChildItem 'C:\Program Files\Java', 'C:\Program Files\Eclipse Adoptium', + 'C:\Program Files\Microsoft', 'C:\Program Files\Android\Android Studio' ` + -Directory -ErrorAction SilentlyContinue | + Where-Object { Test-Path (Join-Path $_.FullName 'bin\javac.exe') } | + Sort-Object Name -Descending | Select-Object -First 1 + if ($j) { $JavaHome = $j.FullName } + } + } +} +if (-not $JavaHome -or -not (Test-Path "$JavaHome\bin\javac.exe")) { + Die "No JDK found. Install a JDK (17 or 21 recommended) and set JAVA_HOME, or pass -JavaHome." +} +$JavaHome = (Resolve-Path $JavaHome).Path +$javaExe = "$JavaHome\bin\java.exe" +$javac = "$JavaHome\bin\javac.exe" +$jar = "$JavaHome\bin\jar.exe" +$keytool = "$JavaHome\bin\keytool.exe" +foreach ($t in $javaExe, $javac, $jar, $keytool) { if (-not (Test-Path $t)) { Die "Missing $t (a JRE is not enough - a full JDK is required)." } } +# apksigner.bat and d8.bat resolve java through JAVA_HOME. +$env:JAVA_HOME = $JavaHome + +# -------------------------------------------------------------------- Kotlin +function Resolve-KotlinHome { + param([string]$Dir) + if (-not $Dir) { return $null } + if (Test-Path (Join-Path $Dir 'bin\kotlinc.bat')) { return (Resolve-Path $Dir).Path } + # Accept the parent of an extracted kotlin-compiler zip too. + $inner = Join-Path $Dir 'kotlinc' + if (Test-Path (Join-Path $inner 'bin\kotlinc.bat')) { return (Resolve-Path $inner).Path } + return $null +} + +$toolsDir = Join-Path $Root '.tools' +if (-not $KotlinHome) { + foreach ($c in $env:KOTLIN_HOME, (Join-Path $toolsDir "kotlinc-$KotlinVersion")) { + $r = Resolve-KotlinHome $c + if ($r) { $KotlinHome = $r; break } + } +} else { + $KotlinHome = Resolve-KotlinHome $KotlinHome + if (-not $KotlinHome) { Die "No bin\kotlinc.bat under the -KotlinHome you passed." } +} + +if (-not $KotlinHome) { + if ($NoDownload) { Die "Kotlin compiler not found. Set KOTLIN_HOME or drop -NoDownload." } + $dest = Join-Path $toolsDir "kotlinc-$KotlinVersion" + $zip = Join-Path $toolsDir "kotlin-compiler-$KotlinVersion.zip" + $url = "https://github.com/JetBrains/kotlin/releases/download/v$KotlinVersion/kotlin-compiler-$KotlinVersion.zip" + New-Item -ItemType Directory -Force -Path $toolsDir | Out-Null + Step "Downloading Kotlin $KotlinVersion (~85 MB, one time)..." + try { + Invoke-WebRequest -Uri $url -OutFile $zip -UseBasicParsing + } catch { + Die "Kotlin download failed: $($_.Exception.Message)`nDownload $url manually, extract it, and pass -KotlinHome \kotlinc." + } + Step 'Extracting Kotlin...' + if (Test-Path $dest) { Remove-Item $dest -Recurse -Force } + Add-Type -AssemblyName System.IO.Compression.FileSystem + [System.IO.Compression.ZipFile]::ExtractToDirectory($zip, $dest) + Remove-Item $zip -Force + $KotlinHome = Resolve-KotlinHome $dest + if (-not $KotlinHome) { Die "Extracted Kotlin to $dest but found no bin\kotlinc.bat." } +} + +# kotlinc.bat is deliberately NOT used: cmd.exe treats ';' as an argument +# delimiter, which shreds any -classpath we hand it. Drive the compiler jar directly. +$kotlinCompilerJar = Join-Path $KotlinHome 'lib\kotlin-compiler.jar' +if (-not (Test-Path $kotlinCompilerJar)) { Die "Missing $kotlinCompilerJar." } +$coro = Join-Path $KotlinHome 'lib\kotlinx-coroutines-core-jvm.jar' +$stdlib = @('kotlin-stdlib.jar', 'kotlin-stdlib-jdk7.jar', 'kotlin-stdlib-jdk8.jar') | + ForEach-Object { Join-Path $KotlinHome "lib\$_" } | Where-Object { Test-Path $_ } +if (-not (Test-Path $coro)) { Die "Missing $coro - use a Kotlin distribution that bundles kotlinx-coroutines-core-jvm.jar." } +if (-not $stdlib) { Die "No kotlin-stdlib jars under $KotlinHome\lib." } + +# ----------------------------------------------------------------- native lib +$libDir = Join-Path $Root 'lib\arm64-v8a' +$so = Join-Path $libDir 'libdragontcp_client.so' +if ($BuildCore -or -not (Test-Path $so)) { + $why = if ($BuildCore) { '-BuildCore was requested' } else { 'libdragontcp_client.so is missing' } + if (-not $RepoRoot) { Die "$why but no core\go.mod was found above $Root. Pass -RepoRoot." } + $coreScript = Join-Path $RepoRoot 'build_core.ps1' + if (-not (Test-Path $coreScript)) { Die "$why but $coreScript was not found. Pass -RepoRoot." } + Step 'Native core (Go)...' + & powershell.exe -NoProfile -ExecutionPolicy Bypass -File $coreScript -ClientOnly -AndroidLibDir $libDir + if ($LASTEXITCODE -ne 0) { Die 'build_core.ps1 failed.' } +} +if (-not (Test-Path $so)) { Die "Missing $so - build the Go core first." } + +Write-Host "app dir : $Root" +Write-Host "repo root : $(if ($RepoRoot) { $RepoRoot } else { '' })" +Write-Host "SDK : $SdkRoot" +Write-Host "build-tools: $BuildTools" +Write-Host "platform : $Platform" +Write-Host "JDK : $JavaHome" +Write-Host "Kotlin : $KotlinHome" + +# -------------------------------------------------------------------- build +$B = Join-Path $Root 'build' +if (Test-Path $B) { Remove-Item $B -Recurse -Force } +New-Item -ItemType Directory -Force -Path "$B\kclasses", "$B\jclasses", "$B\dex" | Out-Null + +Step 'Resources...' +Invoke-Tool (Join-Path $BT 'aapt.exe') @( + 'package', '-f', + '-M', (Join-Path $Root 'AndroidManifest.xml'), + '-S', (Join-Path $Root 'res'), + '-A', (Join-Path $Root 'assets'), + '-I', $AJ, + '-F', "$B\resources.ap_" +) 'aapt' + +Step 'Kotlin TUN adapter...' +$ktSources = Get-ChildItem "$Root\src" -Recurse -Filter *.kt | ForEach-Object { $_.FullName } +if (-not $ktSources) { Die "No .kt sources under $Root\src." } +$CP = (@($AJ, $coro) + $stdlib) -join ';' +Invoke-Tool $javaExe (@( + '-Xmx2g', '-Dfile.encoding=UTF-8', '-Djava.awt.headless=true', + '-cp', $kotlinCompilerJar, 'org.jetbrains.kotlin.cli.jvm.K2JVMCompiler', + '-nowarn', '-no-stdlib', '-jvm-target', '1.8', + '-kotlin-home', $KotlinHome, + '-classpath', $CP, '-d', "$B\kclasses" +) + $ktSources) 'kotlinc' + +Step 'Java UI/service...' +$javaSources = Get-ChildItem "$Root\src" -Recurse -Filter *.java | ForEach-Object { $_.FullName } +if (-not $javaSources) { Die "No .java sources under $Root\src." } +$JCP = (@($AJ, $coro, "$B\kclasses") + $stdlib) -join ';' +# --release 8 keeps the API surface at Java 8; -bootclasspath is rejected by modern javac. +Invoke-Tool $javac (@('-nowarn', '--release', '8', '-classpath', $JCP, '-d', "$B\jclasses") + $javaSources) 'javac' + +Invoke-Tool $jar @('cf', "$B\kclasses.jar", '-C', "$B\kclasses", '.') 'jar (kotlin)' +Invoke-Tool $jar @('cf', "$B\jclasses.jar", '-C', "$B\jclasses", '.') 'jar (java)' + +Step 'DEX...' +Invoke-Tool (Join-Path $BT 'd8.bat') (@( + '--lib', $AJ, '--min-api', '29', '--output', "$B\dex", + "$B\kclasses.jar", "$B\jclasses.jar", $coro +) + $stdlib) 'd8' + +Step 'Packaging...' +$unsigned = "$B\DragonTCP-Hybrid-unsigned.apk" +Copy-Item "$B\resources.ap_" $unsigned -Force + +Add-Type -AssemblyName System.IO.Compression.FileSystem +$zipFile = [System.IO.Compression.ZipFile]::Open($unsigned, 'Update') +try { + $level = [System.IO.Compression.CompressionLevel]::Optimal + foreach ($dex in Get-ChildItem "$B\dex" -Filter 'classes*.dex') { + [System.IO.Compression.ZipFileExtensions]::CreateEntryFromFile($zipFile, $dex.FullName, $dex.Name, $level) | Out-Null + } + $libRoot = Join-Path $Root 'lib' + foreach ($f in Get-ChildItem $libRoot -Recurse -File) { + $entry = 'lib/' + $f.FullName.Substring($libRoot.Length + 1).Replace('\', '/') + [System.IO.Compression.ZipFileExtensions]::CreateEntryFromFile($zipFile, $f.FullName, $entry, $level) | Out-Null + } +} finally { + $zipFile.Dispose() +} + +Step 'Aligning...' +$aligned = "$B\DragonTCP-Hybrid-aligned.apk" +Invoke-Tool (Join-Path $BT 'zipalign.exe') @('-p', '-f', '4', $unsigned, $aligned) 'zipalign' + +# ------------------------------------------------------------------- signing +if (-not $Keystore) { $Keystore = Join-Path $Root 'dragontcp-lite-debug.jks' } +if (-not (Test-Path $Keystore)) { + Step 'Generating debug keystore...' + Invoke-Tool $keytool @( + '-genkeypair', '-keystore', $Keystore, '-storepass', $KsPass, '-keypass', $KeyPass, + '-alias', $KeyAlias, '-keyalg', 'RSA', '-keysize', '2048', '-validity', '10000', + '-dname', 'CN=DragonTCP Lite,O=DragonTCP,C=US' + ) 'keytool' +} + +Step 'Signing...' +$out = "$B\DragonTCP-Hybrid-arm64.apk" +Invoke-Tool (Join-Path $BT 'apksigner.bat') @( + 'sign', '--ks', $Keystore, '--ks-pass', "pass:$KsPass", '--key-pass', "pass:$KeyPass", + '--out', $out, $aligned +) 'apksigner sign' +Invoke-Tool (Join-Path $BT 'apksigner.bat') @('verify', '--verbose', $out) 'apksigner verify' + +Write-Host '' +Write-Host "APK: $out" -ForegroundColor Green +Write-Host ("Size: {0:N1} MB" -f ((Get-Item $out).Length / 1MB)) +Write-Host "Install with: adb install -r `"$out`"" diff --git a/android/android/build_apk.sh b/android/build_apk.sh similarity index 100% rename from android/android/build_apk.sh rename to android/build_apk.sh diff --git a/android/build_core.sh b/android/build_core.sh deleted file mode 100644 index c8b9fea..0000000 --- a/android/build_core.sh +++ /dev/null @@ -1,21 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail -ROOT="$(cd "$(dirname "$0")" && pwd)" -GO_BIN="${GO_BIN:-go}" -command -v "$GO_BIN" >/dev/null 2>&1 || { echo "Go compiler not found" >&2; exit 1; } -mkdir -p "$ROOT/bin" "$ROOT/android/lib/arm64-v8a" -cd "$ROOT/core" - -echo "[core] Android ARM64 client..." -CGO_ENABLED=0 GOOS=android GOARCH=arm64 "$GO_BIN" build -trimpath -ldflags='-s -w' \ - -o "$ROOT/android/lib/arm64-v8a/libdragontcp_client.so" ./cmd/dragontcp-client - -echo "[core] Linux AMD64 server..." -CGO_ENABLED=0 GOOS=linux GOARCH=amd64 "$GO_BIN" build -trimpath -ldflags='-s -w' \ - -o "$ROOT/bin/dragontcp-hybrid-server-linux-amd64" ./cmd/dragontcp-server - -echo "[core] Linux ARM64 server..." -CGO_ENABLED=0 GOOS=linux GOARCH=arm64 "$GO_BIN" build -trimpath -ldflags='-s -w' \ - -o "$ROOT/bin/dragontcp-hybrid-server-linux-arm64" ./cmd/dragontcp-server - -echo "Core build complete." diff --git a/android/core/cmd/dragontcp-client/chunk.go b/android/core/cmd/dragontcp-client/chunk.go deleted file mode 100644 index f68c9d4..0000000 --- a/android/core/cmd/dragontcp-client/chunk.go +++ /dev/null @@ -1,791 +0,0 @@ -package main - -import ( - "crypto/rand" - "encoding/binary" - "fmt" - "io" - "net" - "sort" - "sync" - "sync/atomic" - "time" - - "dragontcp/internal/protocol" - "dragontcp/internal/wire" -) - -type chunkClientOptions struct { - startSize int - minSize int - maxSize int - adaptive bool - adaptSuccesses int - adaptLog bool - pollers int - reconnectEvery int - pollDelay time.Duration - txnTimeout time.Duration - tcpBuffer int -} - -type adaptiveSizer struct { - mu sync.Mutex - name string - current int - min int - max int - adaptive bool - adaptSuccesses int - successes int - good int - bad int - logChanges bool -} - -func newAdaptiveSizer(name string, start int, opts chunkClientOptions) *adaptiveSizer { - if start < opts.minSize { - start = opts.minSize - } - if start > opts.maxSize { - start = opts.maxSize - } - return &adaptiveSizer{ - name: name, - current: start, - min: opts.minSize, - max: opts.maxSize, - adaptive: opts.adaptive, - adaptSuccesses: func() int { - if opts.adaptSuccesses > 0 { - return opts.adaptSuccesses - } - return 64 - }(), - logChanges: opts.adaptLog, - } -} - -func (s *adaptiveSizer) Current() int { - s.mu.Lock() - defer s.mu.Unlock() - return s.current -} - -func (s *adaptiveSizer) Success(attempted int) { - s.mu.Lock() - defer s.mu.Unlock() - if !s.adaptive || attempted != s.current || s.current >= s.max { - return - } - if attempted > s.good { - s.good = attempted - } - s.successes++ - growAfter := s.adaptSuccesses - if s.bad > 0 && s.bad-s.good <= 64 { - growAfter *= 8 - } - if s.successes < growAfter { - return - } - s.successes = 0 - - old := s.current - next := 0 - if s.bad > old+1 { - next = old + (s.bad-old)/2 - } else { - if s.bad > 0 { - s.bad = 0 - } - step := old / 4 - if step < 32 { - step = 32 - } - next = old + step - } - if next > s.max { - next = s.max - } - if next <= old { - return - } - s.current = next - if s.logChanges { - fmt.Printf("adaptive %s chunk: %d -> %d after stable success\n", s.name, old, next) - } -} - -func (s *adaptiveSizer) Failure(attempted int) (int, int) { - s.mu.Lock() - defer s.mu.Unlock() - old := s.current - if !s.adaptive || attempted != s.current { - return old, old - } - s.successes = 0 - if s.bad == 0 || attempted < s.bad { - s.bad = attempted - } - next := attempted / 2 - if s.good > 0 && s.good < attempted { - next = s.good - } else { - s.good = 0 - } - if next < s.min { - next = s.min - } - if next >= attempted && attempted > s.min { - next = attempted - 1 - } - if next < s.min { - next = s.min - } - s.current = next - if s.logChanges && old != next { - fmt.Printf("adaptive %s chunk: %d -> %d after transport failure\n", s.name, old, next) - } - return old, next -} - -type physicalConn struct { - conn net.Conn - requests int -} - -type requestLane struct { - mu sync.Mutex - serverAddr string - tcpBuffer int - reconnectEvery int - timeout time.Duration - pc *physicalConn - closed bool -} - -func newRequestLane(serverAddr string, tcpBuffer, reconnectEvery int, timeout time.Duration) *requestLane { - return &requestLane{ - serverAddr: serverAddr, - tcpBuffer: tcpBuffer, - reconnectEvery: reconnectEvery, - timeout: timeout, - } -} - -func (l *requestLane) discardLocked() { - if l.pc != nil { - _ = l.pc.conn.Close() - l.pc = nil - } -} - -func (l *requestLane) closeAfterLocked() { - if l.pc != nil && l.reconnectEvery > 0 && l.pc.requests >= l.reconnectEvery { - l.discardLocked() - } -} - -func (l *requestLane) ensureLocked() error { - if l.closed { - return net.ErrClosed - } - if l.pc != nil { - if l.reconnectEvery <= 0 || l.pc.requests < l.reconnectEvery { - return nil - } - l.discardLocked() - } - d := net.Dialer{Timeout: 10 * time.Second, KeepAlive: 30 * time.Second} - conn, err := d.Dial("tcp", l.serverAddr) - if err != nil { - return err - } - protocol.TuneTCP(conn) - protocol.TuneTCPBuffer(conn, l.tcpBuffer) - l.pc = &physicalConn{conn: conn} - return nil -} - -func (l *requestLane) Close() { - l.mu.Lock() - l.closed = true - l.discardLocked() - l.mu.Unlock() -} - -func (l *requestLane) single(mode byte, sid wire.SessionID, seq uint64, payload []byte) (byte, []byte, error) { - l.mu.Lock() - defer l.mu.Unlock() - if err := l.ensureLocked(); err != nil { - return 0, nil, err - } - timeout := l.timeout - if timeout <= 0 { - timeout = 5 * time.Second - } - _ = l.pc.conn.SetDeadline(time.Now().Add(timeout)) - if err := wire.WriteRequest(l.pc.conn, mode, sid, seq, payload); err != nil { - l.discardLocked() - return 0, nil, err - } - status, body, err := wire.ReadResponse(l.pc.conn) - if err != nil { - l.discardLocked() - return 0, nil, err - } - l.pc.requests++ - _ = l.pc.conn.SetDeadline(time.Time{}) - l.closeAfterLocked() - if status != wire.StatusError && len(body) > 0 { - body = wire.DecodeMaskedResponse(status, body, sid, mode, seq) - } - return status, body, nil -} - -// download sends one compact request and consumes up to count response records. -// startOffset is also the response keystream sequence. Each DATA response advances -// it by exactly the returned byte count. -func (l *requestLane) download(sid wire.SessionID, startOffset, ackOffset uint64, maxChunk, count int) ([][]byte, byte, error) { - l.mu.Lock() - defer l.mu.Unlock() - if err := l.ensureLocked(); err != nil { - return nil, 0, err - } - timeout := l.timeout - if timeout <= 0 { - timeout = 5 * time.Second - } - _ = l.pc.conn.SetDeadline(time.Now().Add(timeout)) - - payload := make([]byte, 14) - binary.BigEndian.PutUint64(payload[0:8], ackOffset) - binary.BigEndian.PutUint32(payload[8:12], uint32(maxChunk)) - binary.BigEndian.PutUint16(payload[12:14], uint16(count)) - if err := wire.WriteRequest(l.pc.conn, wire.ModeDownload, sid, startOffset, payload); err != nil { - l.discardLocked() - return nil, 0, err - } - - out := make([][]byte, 0, count) - offset := startOffset - lastStatus := wire.StatusOK - for i := 0; i < count; i++ { - status, body, err := wire.ReadResponse(l.pc.conn) - if err != nil { - l.discardLocked() - return out, lastStatus, err - } - lastStatus = status - switch status { - case wire.StatusData: - body = wire.DecodeMaskedResponse(status, body, sid, wire.ModeDownload, offset) - if len(body) == 0 { - l.discardLocked() - return out, status, fmt.Errorf("empty DATA response") - } - out = append(out, body) - offset += uint64(len(body)) - case wire.StatusWait, wire.StatusEOF: - i = count // stop after this response - case wire.StatusError: - l.discardLocked() - return out, status, fmt.Errorf("%s", string(body)) - default: - l.discardLocked() - return out, status, fmt.Errorf("unknown response status %d", status) - } - if status == wire.StatusWait || status == wire.StatusEOF { - break - } - } - - l.pc.requests++ - _ = l.pc.conn.SetDeadline(time.Time{}) - l.closeAfterLocked() - return out, lastStatus, nil -} - -type pathProfile struct { - upload int - download int - persistent bool - at time.Time -} - -var profileState struct { - sync.Mutex - key string - p pathProfile -} - -var probeSeq atomic.Uint64 - -func randomSessionID() (wire.SessionID, error) { - var sid wire.SessionID - _, err := rand.Read(sid[:]) - return sid, err -} - -func probePattern(n int) []byte { - out := make([]byte, n) - for i := range out { - out[i] = byte((i*31 + 17) & 0xff) - } - return out -} - -func makeProbePayload(kind byte, value, total int, token string) []byte { - base := 11 + len(token) - if total < base { - total = base - } - out := make([]byte, total) - copy(out[:4], wire.ProbeMagic[:]) - out[4] = kind - binary.BigEndian.PutUint16(out[5:7], uint16(len(token))) - binary.BigEndian.PutUint32(out[7:11], uint32(value)) - copy(out[11:11+len(token)], token) - for i := base; i < len(out); i++ { - out[i] = byte((i*31 + 17) & 0xff) - } - return out -} - -func probeOne(serverAddr, token string, opts chunkClientOptions, kind byte, candidate int) bool { - sid, err := randomSessionID() - if err != nil { - return false - } - timeout := opts.txnTimeout - if timeout <= 0 || timeout > 2500*time.Millisecond { - timeout = 2500 * time.Millisecond - } - lane := newRequestLane(serverAddr, opts.tcpBuffer, 1, timeout) - defer lane.Close() - seq := probeSeq.Add(1) - - total := 0 - value := candidate - if kind == wire.ProbeUpload { - total = candidate - } - payload := makeProbePayload(kind, value, total, token) - status, body, err := lane.single(wire.ModeProbe, sid, seq, payload) - if err != nil { - return false - } - if kind == wire.ProbeUpload { - return status == wire.StatusOK - } - if kind == wire.ProbeDownload { - if status != wire.StatusData || len(body) != candidate { - return false - } - want := probePattern(candidate) - for i := range body { - if body[i] != want[i] { - return false - } - } - return true - } - return status == wire.StatusOK -} - -func probePersistent(serverAddr, token string, opts chunkClientOptions) bool { - sid, err := randomSessionID() - if err != nil { - return false - } - timeout := opts.txnTimeout - if timeout <= 0 || timeout > 2500*time.Millisecond { - timeout = 2500 * time.Millisecond - } - lane := newRequestLane(serverAddr, opts.tcpBuffer, 0, timeout) - defer lane.Close() - for i := 0; i < 8; i++ { - seq := probeSeq.Add(1) - payload := makeProbePayload(wire.ProbeKeepalive, i, 32+len(token), token) - status, _, err := lane.single(wire.ModeProbe, sid, seq, payload) - if err != nil || status != wire.StatusOK { - return false - } - } - return true -} - -func probeCandidates(minSize, maxSize int) []int { - base := []int{32, 64, 128, 256, 512, 1024, 1200, 1280, 1320, 1350, 1360, 1380, 1400, 1450, 1600, 2048, 3205, 4096, 8192, 16384, 32768, 65536, 98304, 131072, 262144, 524288, 786432, 1048576} - seen := map[int]bool{} - out := make([]int, 0, len(base)+2) - for _, n := range base { - if n >= minSize && n <= maxSize && !seen[n] { - out = append(out, n) - seen[n] = true - } - } - if !seen[minSize] { - out = append(out, minSize) - } - if !seen[maxSize] { - out = append(out, maxSize) - } - sort.Ints(out) - return out -} - -func probeMaximum(serverAddr, token string, opts chunkClientOptions, kind byte) int { - candidates := probeCandidates(opts.minSize, opts.maxSize) - lo, hi := 0, len(candidates)-1 - best := opts.minSize - for lo <= hi { - mid := lo + (hi-lo)/2 - candidate := candidates[mid] - if probeOne(serverAddr, token, opts, kind, candidate) { - best = candidate - lo = mid + 1 - } else { - hi = mid - 1 - } - } - if best < opts.minSize { - best = opts.minSize - } - return best -} - -func getPathProfile(serverAddr, token string, opts chunkClientOptions) pathProfile { - key := fmt.Sprintf("%s|%s|%d|%d", serverAddr, token, opts.minSize, opts.maxSize) - profileState.Lock() - if profileState.key == key && time.Since(profileState.p.at) < 30*time.Minute { - p := profileState.p - profileState.Unlock() - return p - } - profileState.Unlock() - - fallbackUp := minInt(opts.maxSize, maxInt(opts.minSize, 32768)) - fallbackDown := minInt(opts.maxSize, maxInt(opts.minSize, 1350)) - - upCh := make(chan int, 1) - downCh := make(chan int, 1) - go func() { upCh <- probeMaximum(serverAddr, token, opts, wire.ProbeUpload) }() - go func() { downCh <- probeMaximum(serverAddr, token, opts, wire.ProbeDownload) }() - - p := pathProfile{upload: fallbackUp, download: fallbackDown, persistent: false, at: time.Now()} - select { - case p.upload = <-upCh: - case <-time.After(20 * time.Second): - } - select { - case p.download = <-downCh: - case <-time.After(20 * time.Second): - } - p.persistent = probePersistent(serverAddr, token, opts) - - fmt.Printf("path probe: upload=%d download=%d persistent=%t\n", p.upload, p.download, p.persistent) - - profileState.Lock() - profileState.key = key - profileState.p = p - profileState.Unlock() - return p -} - -func encodeOpen(token, host string, port int) ([]byte, error) { - if len(token) > 65535 || len(host) > 65535 { - return nil, fmt.Errorf("token or hostname too long") - } - out := make([]byte, 6+len(token)+len(host)) - binary.BigEndian.PutUint16(out[0:2], uint16(len(token))) - binary.BigEndian.PutUint16(out[2:4], uint16(len(host))) - binary.BigEndian.PutUint16(out[4:6], uint16(port)) - copy(out[6:6+len(token)], token) - copy(out[6+len(token):], host) - return out, nil -} - -type chunkConn struct { - sid wire.SessionID - opts chunkClientOptions - uploadLane *requestLane - downloadLane *requestLane - serverMax int - upSizer *adaptiveSizer - downSizer *adaptiveSizer - - writeMu sync.Mutex - upOffset uint64 - - readMu sync.Mutex - readBuf []byte - downloadOffset uint64 - consumedOffset uint64 - eof bool - pipeline int - maxPipeline int - - closeOnce sync.Once -} - -func openChunkTunnel(serverAddr, token, targetHost string, targetPort int, opts chunkClientOptions) (net.Conn, error) { - if opts.minSize < 32 { - opts.minSize = 32 - } - if opts.maxSize < opts.minSize { - opts.maxSize = opts.minSize - } - if opts.maxSize > 1024*1024 { - opts.maxSize = 1024 * 1024 - } - if opts.txnTimeout <= 0 { - opts.txnTimeout = 5 * time.Second - } - if opts.adaptSuccesses < 1 { - opts.adaptSuccesses = 64 - } - if opts.reconnectEvery < 0 { - opts.reconnectEvery = 0 - } - - profile := getPathProfile(serverAddr, token, opts) - reconnect := opts.reconnectEvery - // Compatibility-friendly reconnect modes: - // 0 = persistent (CLI explicit) - // 1 = auto: persistent when the path probe succeeds, otherwise one request/connection - // N>=2 = force connection rotation after N logical requests - if reconnect == 1 { - if profile.persistent { - reconnect = 0 - fmt.Printf("path probe: reconnect mode auto -> persistent\n") - } else { - fmt.Printf("path probe: reconnect mode auto -> every request\n") - } - } - - sid, err := randomSessionID() - if err != nil { - return nil, err - } - control := newRequestLane(serverAddr, opts.tcpBuffer, reconnect, opts.txnTimeout) - payload, err := encodeOpen(token, targetHost, targetPort) - if err != nil { - control.Close() - return nil, err - } - status, body, err := control.single(wire.ModeOpen, sid, 0, payload) - if err != nil { - control.Close() - return nil, err - } - if status == wire.StatusError { - control.Close() - return nil, fmt.Errorf("%s", string(body)) - } - if status != wire.StatusOK || len(body) != 4 { - control.Close() - return nil, fmt.Errorf("bad OPEN response") - } - serverMax := int(binary.BigEndian.Uint32(body)) - control.Close() - if serverMax < opts.minSize { - return nil, fmt.Errorf("server maximum chunk %d is below client minimum %d", serverMax, opts.minSize) - } - if opts.maxSize > serverMax { - opts.maxSize = serverMax - } - upStart := minInt(profile.upload, opts.maxSize) - downStart := minInt(profile.download, opts.maxSize) - if upStart < opts.minSize { - upStart = opts.minSize - } - if downStart < opts.minSize { - downStart = opts.minSize - } - - c := &chunkConn{ - sid: sid, - opts: opts, - serverMax: serverMax, - uploadLane: newRequestLane(serverAddr, opts.tcpBuffer, reconnect, opts.txnTimeout), - downloadLane: newRequestLane(serverAddr, opts.tcpBuffer, reconnect, opts.txnTimeout), - // BHTTP-style safe pipeline: request up to 256 records immediately. - // Hybrid v1 already supported 256 on the wire/server; starting at 32 - // made tiny-path downloads spend many RTTs ramping up. - pipeline: 256, - maxPipeline: 256, - } - c.upSizer = newAdaptiveSizer("upload", upStart, opts) - c.downSizer = newAdaptiveSizer("download", downStart, opts) - return c, nil -} - -func (c *chunkConn) fillReadBuffer() error { - if c.eof { - return io.EOF - } - minFailures := 0 - for len(c.readBuf) == 0 && !c.eof { - chunk := c.downSizer.Current() - count := c.pipeline - if count < 1 { - count = 1 - } - if count > c.maxPipeline { - count = c.maxPipeline - } - // Bound each batch to roughly 1 MiB of useful data. - if maxCount := (1024 * 1024) / maxInt(chunk, 1); maxCount < count { - count = maxInt(maxCount, 1) - } - - data, status, err := c.downloadLane.download(c.sid, c.downloadOffset, c.consumedOffset, chunk, count) - for _, part := range data { - c.readBuf = append(c.readBuf, part...) - c.downloadOffset += uint64(len(part)) - } - if len(data) > 0 { - c.downSizer.Success(chunk) - if c.pipeline < c.maxPipeline { - c.pipeline++ - } - minFailures = 0 - } - if err != nil { - if c.pipeline > 1 { - old := c.pipeline - c.pipeline /= 2 - if c.pipeline < 1 { - c.pipeline = 1 - } - if c.opts.adaptLog && old != c.pipeline { - fmt.Printf("adaptive download pipeline: %d -> %d after transport failure\n", old, c.pipeline) - } - } else { - old, next := c.downSizer.Failure(chunk) - if old == next && next == c.opts.minSize { - minFailures++ - if minFailures >= 8 { - return fmt.Errorf("download failed at minimum chunk %d: %w", next, err) - } - } - } - time.Sleep(30 * time.Millisecond) - if len(c.readBuf) > 0 { - return nil - } - continue - } - switch status { - case wire.StatusEOF: - c.eof = true - case wire.StatusWait: - if c.opts.pollDelay > 0 { - time.Sleep(c.opts.pollDelay) - } else { - time.Sleep(5 * time.Millisecond) - } - } - if len(c.readBuf) > 0 { - return nil - } - } - if c.eof && len(c.readBuf) == 0 { - return io.EOF - } - return nil -} - -func (c *chunkConn) Read(p []byte) (int, error) { - c.readMu.Lock() - defer c.readMu.Unlock() - if len(p) == 0 { - return 0, nil - } - if len(c.readBuf) == 0 { - if err := c.fillReadBuffer(); err != nil { - return 0, err - } - } - n := copy(p, c.readBuf) - c.readBuf = c.readBuf[n:] - c.consumedOffset += uint64(n) - return n, nil -} - -func (c *chunkConn) Write(p []byte) (int, error) { - c.writeMu.Lock() - defer c.writeMu.Unlock() - if len(p) == 0 { - return 0, nil - } - total := 0 - minFailures := 0 - for len(p) > 0 { - size := c.upSizer.Current() - n := minInt(size, len(p)) - status, body, err := c.uploadLane.single(wire.ModeUpload, c.sid, c.upOffset, p[:n]) - if err != nil { - old, next := c.upSizer.Failure(size) - if old == next && next == c.opts.minSize { - minFailures++ - if minFailures >= 8 { - return total, fmt.Errorf("upload failed at minimum chunk %d: %w", next, err) - } - } else { - minFailures = 0 - } - time.Sleep(30 * time.Millisecond) - continue - } - if status == wire.StatusError { - return total, fmt.Errorf("%s", string(body)) - } - if status != wire.StatusOK { - return total, fmt.Errorf("unexpected upload status %d", status) - } - c.upOffset += uint64(n) - total += n - p = p[n:] - c.upSizer.Success(size) - minFailures = 0 - } - return total, nil -} - -func (c *chunkConn) Close() error { - c.closeOnce.Do(func() { - lane := newRequestLane(c.uploadLane.serverAddr, c.opts.tcpBuffer, 1, c.opts.txnTimeout) - _, _, _ = lane.single(wire.ModeClose, c.sid, 0, nil) - lane.Close() - c.uploadLane.Close() - c.downloadLane.Close() - }) - return nil -} - -func (c *chunkConn) LocalAddr() net.Addr { return dummyAddr("dragontcp-binary-local") } -func (c *chunkConn) RemoteAddr() net.Addr { return dummyAddr("dragontcp-binary-remote") } -func (c *chunkConn) SetDeadline(time.Time) error { return nil } -func (c *chunkConn) SetReadDeadline(time.Time) error { return nil } -func (c *chunkConn) SetWriteDeadline(time.Time) error { return nil } - -type dummyAddr string - -func (d dummyAddr) Network() string { return "dragontcp-binary" } -func (d dummyAddr) String() string { return string(d) } - -func minInt(a, b int) int { - if a < b { - return a - } - return b -} -func maxInt(a, b int) int { - if a > b { - return a - } - return b -} diff --git a/android/core/cmd/dragontcp-client/chunk_test.go b/android/core/cmd/dragontcp-client/chunk_test.go deleted file mode 100644 index 16f0a24..0000000 --- a/android/core/cmd/dragontcp-client/chunk_test.go +++ /dev/null @@ -1,31 +0,0 @@ -package main - -import "testing" - -func TestAdaptiveSizerRecoversFromMinimum(t *testing.T) { - opts := chunkClientOptions{ - startSize: 64, - minSize: 32, - maxSize: 1024, - adaptive: true, - adaptSuccesses: 2, - } - s := newAdaptiveSizer("test", 64, opts) - _, next := s.Failure(64) - if next != 32 { - t.Fatalf("failure should reduce 64 -> 32, got %d", next) - } - for i := 0; i < 16; i++ { - s.Success(32) - } - if got := s.Current(); got <= 32 { - t.Fatalf("adaptive controller remained stuck at minimum: %d", got) - } -} - -func TestReconnectZeroMeansPersistent(t *testing.T) { - lane := newRequestLane("127.0.0.1:1", 0, 0, 0) - if lane.reconnectEvery != 0 { - t.Fatalf("reconnectEvery=%d, want 0", lane.reconnectEvery) - } -} diff --git a/android/core/cmd/dragontcp-client/main.go b/android/core/cmd/dragontcp-client/main.go deleted file mode 100644 index a503fef..0000000 --- a/android/core/cmd/dragontcp-client/main.go +++ /dev/null @@ -1,482 +0,0 @@ -package main - -import ( - "bytes" - "flag" - "fmt" - "net" - "net/url" - "os" - "strconv" - "strings" - "sync/atomic" - "time" - - "dragontcp/internal/protocol" -) - -const maxHeader = 128 * 1024 - -var requestCounter atomic.Uint32 - -func readHTTPHeaders(conn net.Conn) ([]byte, []byte, error) { - buf := make([]byte, 0, 8192) - tmp := make([]byte, 8192) - - for { - n, err := conn.Read(tmp) - if n > 0 { - buf = append(buf, tmp[:n]...) - - if len(buf) > maxHeader { - return nil, nil, fmt.Errorf("HTTP headers too large") - } - - if i := bytes.Index(buf, []byte("\r\n\r\n")); i >= 0 { - end := i + 4 - return buf[:end], buf[end:], nil - } - } - - if err != nil { - return nil, nil, err - } - } -} - -func parseHostPort(authority string, defaultPort int) (string, int, error) { - authority = strings.TrimSpace(authority) - - if host, portText, err := net.SplitHostPort(authority); err == nil { - port, err := strconv.Atoi(portText) - return host, port, err - } - - // Host without port. - if strings.HasPrefix(authority, "[") && strings.HasSuffix(authority, "]") { - return strings.Trim(authority, "[]"), defaultPort, nil - } - - if strings.Count(authority, ":") == 0 { - return authority, defaultPort, nil - } - - // Bare IPv6. - if ip := net.ParseIP(authority); ip != nil { - return authority, defaultPort, nil - } - - return "", 0, fmt.Errorf("invalid authority: %s", authority) -} - -func rewritePlainHTTPRequest(header []byte) (string, int, []byte, error) { - text := string(header) - lines := strings.Split(text, "\r\n") - if len(lines) == 0 { - return "", 0, nil, fmt.Errorf("empty request") - } - - parts := strings.SplitN(lines[0], " ", 3) - if len(parts) != 3 { - return "", 0, nil, fmt.Errorf("invalid request line") - } - - method, target, version := parts[0], parts[1], parts[2] - - var ( - hostHeader string - headers []string - ) - - for _, line := range lines[1:] { - if line == "" { - continue - } - - k, v, ok := strings.Cut(line, ":") - if !ok { - continue - } - - lk := strings.ToLower(strings.TrimSpace(k)) - - if lk == "host" { - hostHeader = strings.TrimSpace(v) - } - - if lk == "connection" || - lk == "proxy-connection" || - lk == "proxy-authorization" { - continue - } - - headers = append(headers, k+": "+strings.TrimSpace(v)) - } - - u, err := url.Parse(target) - if err != nil { - return "", 0, nil, err - } - - var host string - var port int - path := target - - if u.Hostname() != "" { - if strings.ToLower(u.Scheme) != "http" { - return "", 0, nil, fmt.Errorf("unsupported plain HTTP scheme: %s", u.Scheme) - } - - host = u.Hostname() - port = 80 - - if u.Port() != "" { - port, err = strconv.Atoi(u.Port()) - if err != nil { - return "", 0, nil, err - } - } - - path = u.EscapedPath() - if path == "" { - path = "/" - } - if u.RawQuery != "" { - path += "?" + u.RawQuery - } - } else { - if hostHeader == "" { - return "", 0, nil, fmt.Errorf("missing Host header") - } - - host, port, err = parseHostPort(hostHeader, 80) - if err != nil { - return "", 0, nil, err - } - if path == "" { - path = "/" - } - } - - var out strings.Builder - fmt.Fprintf(&out, "%s %s %s\r\n", method, path, version) - - sawHost := false - for _, h := range headers { - if strings.HasPrefix(strings.ToLower(h), "host:") { - sawHost = true - } - out.WriteString(h) - out.WriteString("\r\n") - } - - if !sawHost { - if port == 80 { - fmt.Fprintf(&out, "Host: %s\r\n", host) - } else { - fmt.Fprintf(&out, "Host: %s\r\n", net.JoinHostPort(host, strconv.Itoa(port))) - } - } - - out.WriteString("Connection: close\r\n\r\n") - - return host, port, []byte(out.String()), nil -} - -func openDragonTCPTunnel(serverAddr, token, targetHost string, targetPort int, transport string, tcpBuffer int) (net.Conn, error) { - d := net.Dialer{ - Timeout: 10 * time.Second, - KeepAlive: 30 * time.Second, - } - - conn, err := d.Dial("tcp", serverAddr) - if err != nil { - return nil, err - } - - protocol.TuneTCP(conn) - protocol.TuneTCPBuffer(conn, tcpBuffer) - _ = conn.SetDeadline(time.Now().Add(15 * time.Second)) - - // Correlation only; cryptographic randomness is unnecessary here. - requestID := requestCounter.Add(1) - - var command []byte - if transport == "raw" { - command = []byte(fmt.Sprintf("TUNNEL2 %s %s %d RAW", token, targetHost, targetPort)) - } else { - // Legacy XOR command remains compatible with the older server. - command = []byte(fmt.Sprintf("TUNNEL %s %s %d", token, targetHost, targetPort)) - } - - if err := protocol.WriteRequestFrame(conn, requestID, command); err != nil { - conn.Close() - return nil, err - } - - responseID, response, err := protocol.ReadResponseFrame(conn) - if err != nil { - conn.Close() - return nil, err - } - - if responseID != requestID { - conn.Close() - return nil, fmt.Errorf("request ID mismatch") - } - - if string(response) != "CONNECTED" { - conn.Close() - return nil, fmt.Errorf("%s", response) - } - - _ = conn.SetDeadline(time.Time{}) - return conn, nil -} - -func writeHTTPError(conn net.Conn, code int, reason, detail string) { - if detail == "" { - detail = reason - } - - body := []byte(detail) - - fmt.Fprintf( - conn, - "HTTP/1.1 %d %s\r\nContent-Type: text/plain; charset=utf-8\r\nContent-Length: %d\r\nConnection: close\r\n\r\n", - code, - reason, - len(body), - ) - _, _ = conn.Write(body) -} - -func handleLocal(conn net.Conn, serverAddr, token, transport string, tcpBuffer int, chunkOpts chunkClientOptions, slots chan struct{}) { - defer func() { - <-slots - _ = conn.Close() - }() - - protocol.TuneTCP(conn) - protocol.TuneTCPBuffer(conn, tcpBuffer) - _ = conn.SetDeadline(time.Now().Add(15 * time.Second)) - - header, extra, err := readHTTPHeaders(conn) - if err != nil { - return - } - - firstLine := strings.SplitN(string(header), "\r\n", 2)[0] - parts := strings.SplitN(firstLine, " ", 3) - - if len(parts) != 3 { - writeHTTPError(conn, 400, "Bad Request", "invalid HTTP request line") - return - } - - method, target := parts[0], parts[1] - - if strings.EqualFold(method, "CONNECT") { - host, port, err := parseHostPort(target, 443) - if err != nil { - writeHTTPError(conn, 400, "Bad Request", err.Error()) - return - } - - var remote net.Conn - if transport == "chunk" { - remote, err = openChunkTunnel(serverAddr, token, host, port, chunkOpts) - } else { - remote, err = openDragonTCPTunnel(serverAddr, token, host, port, transport, tcpBuffer) - } - if err != nil { - writeHTTPError(conn, 502, "Bad Gateway", err.Error()) - return - } - defer remote.Close() - - _, _ = conn.Write([]byte( - "HTTP/1.1 200 Connection Established\r\n" + - "Proxy-Agent: dragontcp-proxy/2.0\r\n\r\n", - )) - - if len(extra) > 0 { - if transport == "xor" { - protocol.XorInPlace(extra) - } - if _, err := remote.Write(extra); err != nil { - return - } - } - - _ = conn.SetDeadline(time.Time{}) - if transport == "xor" { - protocol.RelayXOR(conn, remote) - } else { - // raw and chunk connections expose a normal plaintext net.Conn. - protocol.RelayRaw(conn, remote) - } - return - } - - host, port, rewritten, err := rewritePlainHTTPRequest(header) - if err != nil { - writeHTTPError(conn, 400, "Bad Request", err.Error()) - return - } - - var remote net.Conn - if transport == "chunk" { - remote, err = openChunkTunnel(serverAddr, token, host, port, chunkOpts) - } else { - remote, err = openDragonTCPTunnel(serverAddr, token, host, port, transport, tcpBuffer) - } - if err != nil { - writeHTTPError(conn, 502, "Bad Gateway", err.Error()) - return - } - defer remote.Close() - - initial := make([]byte, 0, len(rewritten)+len(extra)) - initial = append(initial, rewritten...) - initial = append(initial, extra...) - if transport == "xor" { - protocol.XorInPlace(initial) - } - - if _, err := remote.Write(initial); err != nil { - return - } - - _ = conn.SetDeadline(time.Time{}) - if transport == "xor" { - protocol.RelayXOR(conn, remote) - } else { - protocol.RelayRaw(conn, remote) - } -} - -func main() { - var ( - listenHost = flag.String("listen-host", "127.0.0.1", "local proxy listen host") - listenPort = flag.Int("listen-port", 8080, "local proxy listen port") - serverHost = flag.String("server-host", "", "remote DragonTCP server host") - serverPort = flag.Int("server-port", 53, "remote DragonTCP server port") - token = flag.String("token", "", "optional shared token") - maxConnections = flag.Int("max-connections", 20000, "max simultaneous proxy connections") - transport = flag.String("transport", "chunk", "transport: chunk (DragonTCP binary adaptive transport)") - tcpBuffer = flag.Int("tcp-buffer", 0, "optional TCP read/write buffer bytes; 0 keeps OS autotuning") - chunkStart = flag.Int("chunk-start", 1048576, "initial adaptive chunk payload bytes") - chunkMin = flag.Int("chunk-min", 32, "minimum adaptive chunk payload bytes") - chunkMax = flag.Int("chunk-max", 1048576, "maximum adaptive chunk payload bytes (up to 1 MiB)") - chunkAdaptive = flag.Bool("chunk-adaptive", true, "automatically shrink on failures and grow after stable success") - chunkSuccesses = flag.Int("chunk-grow-after", 16, "successful data records required before increasing chunk size") - chunkAdaptLog = flag.Bool("chunk-adapt-log", true, "print adaptive chunk size changes") - chunkSizeLegacy = flag.Int("chunk-size", 0, "legacy fixed chunk size; nonzero disables adaptation") - chunkPollers = flag.Int("chunk-pollers", 1, "reserved compatibility setting; binary transport uses one download worker") - chunkReconnect = flag.Int("chunk-reconnect-every", 0, "force reconnect after N logical requests; 0 = persistent/automatic") - chunkPollDelay = flag.Duration("chunk-poll-delay", 2*time.Millisecond, "delay after an empty chunk poll") - chunkTimeout = flag.Duration("chunk-timeout", 2*time.Second, "per-record transaction timeout before adaptive shrink") - ) - flag.Parse() - - if *serverHost == "" { - fmt.Fprintln(os.Stderr, "--server-host is required") - os.Exit(2) - } - - *transport = strings.ToLower(*transport) - if *transport != "chunk" { - fmt.Fprintln(os.Stderr, "DragonTCP requires --transport chunk (binary adaptive TCP/53 transport)") - os.Exit(2) - } - if *chunkSizeLegacy != 0 { - if *chunkSizeLegacy < 32 || *chunkSizeLegacy > protocol.MaxChunkPayload { - fmt.Fprintf(os.Stderr, "--chunk-size must be between 32 and %d\n", protocol.MaxChunkPayload) - os.Exit(2) - } - *chunkStart = *chunkSizeLegacy - *chunkMin = *chunkSizeLegacy - *chunkMax = *chunkSizeLegacy - *chunkAdaptive = false - } - if *chunkMin < 32 || *chunkMax > protocol.MaxChunkPayload || *chunkMin > *chunkStart || *chunkStart > *chunkMax { - fmt.Fprintf(os.Stderr, "require 32 <= --chunk-min <= --chunk-start <= --chunk-max <= %d\n", protocol.MaxChunkPayload) - os.Exit(2) - } - if *chunkSuccesses < 1 { - fmt.Fprintln(os.Stderr, "--chunk-grow-after must be at least 1") - os.Exit(2) - } - if *chunkPollers < 1 || *chunkPollers > 128 { - fmt.Fprintln(os.Stderr, "--chunk-pollers must be between 1 and 128") - os.Exit(2) - } - if *chunkReconnect < 0 { - fmt.Fprintln(os.Stderr, "--chunk-reconnect-every must be 0 or greater") - os.Exit(2) - } - chunkOpts := chunkClientOptions{ - startSize: *chunkStart, - minSize: *chunkMin, - maxSize: *chunkMax, - adaptive: *chunkAdaptive, - adaptSuccesses: *chunkSuccesses, - adaptLog: *chunkAdaptLog, - pollers: *chunkPollers, - reconnectEvery: *chunkReconnect, - pollDelay: *chunkPollDelay, - txnTimeout: *chunkTimeout, - tcpBuffer: *tcpBuffer, - } - - listenAddr := net.JoinHostPort(*listenHost, strconv.Itoa(*listenPort)) - serverAddr := net.JoinHostPort(*serverHost, strconv.Itoa(*serverPort)) - - ln, err := net.Listen("tcp", listenAddr) - if err != nil { - fmt.Fprintln(os.Stderr, err) - os.Exit(1) - } - defer ln.Close() - - fmt.Printf("local Go HTTP proxy listening on %s\n", listenAddr) - fmt.Printf("remote DragonTCP endpoint=%s\n", serverAddr) - fmt.Printf("max_connections=%d transport=%s tcp_buffer=%d\n", *maxConnections, *transport, *tcpBuffer) - if *transport == "chunk" { - fmt.Printf( - "adaptive_chunk=%v start=%d min=%d max=%d grow_after=%d pollers=%d reconnect_every=%d timeout=%s\n", - *chunkAdaptive, - *chunkStart, - *chunkMin, - *chunkMax, - *chunkSuccesses, - *chunkPollers, - *chunkReconnect, - chunkTimeout.String(), - ) - } - - slots := make(chan struct{}, *maxConnections) - - for { - conn, err := ln.Accept() - if err != nil { - fmt.Fprintln(os.Stderr, "accept:", err) - continue - } - - select { - case slots <- struct{}{}: - go handleLocal(conn, serverAddr, *token, *transport, *tcpBuffer, chunkOpts, slots) - default: - writeHTTPError( - conn, - 503, - "Service Unavailable", - "proxy connection limit reached", - ) - _ = conn.Close() - } - } -} diff --git a/android/core/cmd/dragontcp-server/chunk.go b/android/core/cmd/dragontcp-server/chunk.go deleted file mode 100644 index a49486e..0000000 --- a/android/core/cmd/dragontcp-server/chunk.go +++ /dev/null @@ -1,507 +0,0 @@ -package main - -import ( - "bytes" - "context" - "encoding/binary" - "fmt" - "net" - "sync" - "time" - - "dragontcp/internal/wire" -) - -type streamSession struct { - sid wire.SessionID - target net.Conn - targetName string - maxChunk int - maxBuffer int - debug *serverDebug - - mu sync.Mutex - notify chan struct{} - buf []byte - base uint64 - eof bool - closed bool - lastSeen time.Time - - upMu sync.Mutex - expectedUp uint64 -} - -func newStreamSession(sid wire.SessionID, target net.Conn, targetName string, maxChunk, maxBuffer int, debug *serverDebug) *streamSession { - s := &streamSession{ - sid: sid, - target: target, - targetName: targetName, - maxChunk: maxChunk, - maxBuffer: maxBuffer, - debug: debug, - notify: make(chan struct{}), - lastSeen: time.Now(), - } - go s.readTarget() - return s -} - -func (s *streamSession) signalLocked() { - close(s.notify) - s.notify = make(chan struct{}) -} - -func (s *streamSession) touchLocked() { s.lastSeen = time.Now() } - -func (s *streamSession) readTarget() { - tmp := make([]byte, 64*1024) - for { - n, err := s.target.Read(tmp) - if n > 0 { - data := append([]byte(nil), tmp[:n]...) - for len(data) > 0 { - s.mu.Lock() - for !s.closed && len(s.buf) >= s.maxBuffer { - ch := s.notify - s.mu.Unlock() - <-ch - s.mu.Lock() - } - if s.closed { - s.mu.Unlock() - return - } - room := s.maxBuffer - len(s.buf) - take := len(data) - if take > room { - take = room - } - s.buf = append(s.buf, data[:take]...) - data = data[take:] - s.touchLocked() - s.signalLocked() - s.mu.Unlock() - if s.debug != nil && s.debug.enabled { - s.debug.bytesDown.Add(uint64(take)) - } - } - } - if err != nil { - s.mu.Lock() - if !s.closed { - s.eof = true - s.touchLocked() - s.signalLocked() - } - s.mu.Unlock() - return - } - } -} - -func (s *streamSession) ackLocked(offset uint64) { - if offset <= s.base { - return - } - end := s.base + uint64(len(s.buf)) - if offset > end { - offset = end - } - drop := int(offset - s.base) - if drop <= 0 { - return - } - s.buf = s.buf[drop:] - s.base = offset - if len(s.buf) == 0 { - s.buf = nil - } else if cap(s.buf) > 4*len(s.buf) && cap(s.buf) > 1024*1024 { - compact := append([]byte(nil), s.buf...) - s.buf = compact - } - s.signalLocked() -} - -func (s *streamSession) ack(offset uint64) { - s.mu.Lock() - s.ackLocked(offset) - s.touchLocked() - s.mu.Unlock() -} - -func (s *streamSession) readAt(offset uint64, limit int, wait time.Duration) ([]byte, byte, error) { - if limit < 1 || limit > s.maxChunk { - return nil, wire.StatusError, fmt.Errorf("invalid download limit %d", limit) - } - deadline := time.Now().Add(wait) - firstDataAt := time.Time{} - - for { - s.mu.Lock() - s.touchLocked() - if offset < s.base { - s.mu.Unlock() - return nil, wire.StatusError, fmt.Errorf("download offset %d was already acknowledged (base=%d)", offset, s.base) - } - rel64 := offset - s.base - if rel64 <= uint64(len(s.buf)) { - rel := int(rel64) - available := len(s.buf) - rel - if available > 0 { - if firstDataAt.IsZero() { - firstDataAt = time.Now() - } - // Coalesce tiny target reads briefly. This prevents a 1-2 byte - // producer read from becoming a permanent tiny tunnel record. - if available < limit && !s.eof && wait > 0 && time.Since(firstDataAt) < 2*time.Millisecond { - ch := s.notify - s.mu.Unlock() - select { - case <-ch: - case <-time.After(2 * time.Millisecond): - } - continue - } - n := available - if n > limit { - n = limit - } - out := append([]byte(nil), s.buf[rel:rel+n]...) - s.mu.Unlock() - return out, wire.StatusData, nil - } - if s.eof || s.closed { - s.mu.Unlock() - return nil, wire.StatusEOF, nil - } - } else { - s.mu.Unlock() - return nil, wire.StatusError, fmt.Errorf("download offset %d is beyond buffered stream end %d", offset, s.base+uint64(len(s.buf))) - } - - if wait <= 0 || time.Now().After(deadline) { - s.mu.Unlock() - return nil, wire.StatusWait, nil - } - ch := s.notify - remaining := time.Until(deadline) - s.mu.Unlock() - select { - case <-ch: - case <-time.After(remaining): - return nil, wire.StatusWait, nil - } - } -} - -func (s *streamSession) upload(offset uint64, data []byte) error { - if len(data) == 0 || len(data) > s.maxChunk { - return fmt.Errorf("invalid upload size %d", len(data)) - } - s.upMu.Lock() - defer s.upMu.Unlock() - - if offset < s.expectedUp { - // Idempotent retry after a lost ACK. - if offset+uint64(len(data)) <= s.expectedUp { - return nil - } - return fmt.Errorf("overlapping upload retry at %d", offset) - } - if offset != s.expectedUp { - return fmt.Errorf("upload gap: got %d expected %d", offset, s.expectedUp) - } - if _, err := s.target.Write(data); err != nil { - return err - } - s.expectedUp += uint64(len(data)) - s.mu.Lock() - s.touchLocked() - s.mu.Unlock() - if s.debug != nil && s.debug.enabled { - s.debug.bytesUp.Add(uint64(len(data))) - s.debug.pushRecords.Add(1) - } - return nil -} - -func (s *streamSession) close() { - s.mu.Lock() - if s.closed { - s.mu.Unlock() - return - } - s.closed = true - s.signalLocked() - s.mu.Unlock() - _ = s.target.Close() -} - -type streamManager struct { - mu sync.RWMutex - sessions map[string]*streamSession - timeout time.Duration - debug *serverDebug -} - -func sidKey(sid wire.SessionID) string { return string(sid[:]) } - -func newStreamManager(timeout time.Duration, debug *serverDebug) *streamManager { - m := &streamManager{sessions: make(map[string]*streamSession), timeout: timeout, debug: debug} - go m.cleanupLoop() - return m -} - -func (m *streamManager) get(sid wire.SessionID) *streamSession { - m.mu.RLock() - s := m.sessions[sidKey(sid)] - m.mu.RUnlock() - return s -} - -func (m *streamManager) addOrGet(sid wire.SessionID, s *streamSession) (*streamSession, bool) { - key := sidKey(sid) - m.mu.Lock() - if old := m.sessions[key]; old != nil { - m.mu.Unlock() - s.close() - return old, false - } - m.sessions[key] = s - m.mu.Unlock() - return s, true -} - -func (m *streamManager) remove(sid wire.SessionID) { - key := sidKey(sid) - m.mu.Lock() - s := m.sessions[key] - delete(m.sessions, key) - m.mu.Unlock() - if s != nil { - s.close() - if m.debug != nil && m.debug.enabled { - m.debug.sessionsClosed.Add(1) - m.debug.activeSessions.Add(-1) - } - } -} - -func (m *streamManager) count() int { m.mu.RLock(); n := len(m.sessions); m.mu.RUnlock(); return n } - -func (m *streamManager) cleanupLoop() { - ticker := time.NewTicker(30 * time.Second) - defer ticker.Stop() - for range ticker.C { - cutoff := time.Now().Add(-m.timeout) - var stale []wire.SessionID - m.mu.RLock() - for _, s := range m.sessions { - s.mu.Lock() - last := s.lastSeen - closed := s.closed - sid := s.sid - s.mu.Unlock() - if closed || last.Before(cutoff) { - stale = append(stale, sid) - } - } - m.mu.RUnlock() - for _, sid := range stale { - m.remove(sid) - } - } -} - -func parseProbe(payload []byte) (kind byte, value int, token string, err error) { - if len(payload) < 11 || !bytes.Equal(payload[:4], wire.ProbeMagic[:]) { - return 0, 0, "", fmt.Errorf("bad probe payload") - } - kind = payload[4] - tl := int(binary.BigEndian.Uint16(payload[5:7])) - value = int(binary.BigEndian.Uint32(payload[7:11])) - if 11+tl > len(payload) { - return 0, 0, "", fmt.Errorf("bad probe token length") - } - token = string(payload[11 : 11+tl]) - return -} - -func probePattern(n int) []byte { - out := make([]byte, n) - for i := range out { - out[i] = byte((i*31 + 17) & 0xff) - } - return out -} - -func parseOpen(payload []byte) (token, host string, port int, err error) { - if len(payload) < 6 { - return "", "", 0, fmt.Errorf("bad OPEN payload") - } - tl := int(binary.BigEndian.Uint16(payload[0:2])) - hl := int(binary.BigEndian.Uint16(payload[2:4])) - port = int(binary.BigEndian.Uint16(payload[4:6])) - if port < 1 || 6+tl+hl != len(payload) { - return "", "", 0, fmt.Errorf("bad OPEN lengths") - } - token = string(payload[6 : 6+tl]) - host = string(payload[6+tl:]) - if host == "" { - return "", "", 0, fmt.Errorf("empty target host") - } - return -} - -func processWireRequest(conn net.Conn, req wire.Request, token string, allowPrivate bool, cache *dnsCache, tcpBuffer int, manager *streamManager, maxChunk, maxBuffer int, pollWait time.Duration, debug *serverDebug) error { - switch req.Mode { - case wire.ModeProbe: - kind, value, supplied, err := parseProbe(req.Payload) - if err != nil { - return wire.WriteResponse(conn, wire.StatusError, []byte(err.Error())) - } - if !tokenEqual(supplied, token) { - return wire.WriteResponse(conn, wire.StatusError, []byte("authentication failed")) - } - switch kind { - case wire.ProbeUpload: - if len(req.Payload) > maxChunk { - return wire.WriteResponse(conn, wire.StatusError, []byte("probe too large")) - } - return wire.WriteResponse(conn, wire.StatusOK, nil) - case wire.ProbeDownload: - if value < 1 || value > maxChunk { - return wire.WriteResponse(conn, wire.StatusError, []byte("probe too large")) - } - return wire.WriteMaskedResponse(conn, wire.StatusData, probePattern(value), req.Session, wire.ModeProbe, req.Seq) - case wire.ProbeKeepalive: - return wire.WriteResponse(conn, wire.StatusOK, nil) - case wire.ProbeBatch: - count := value - if count < 1 { - count = 1 - } - if count > 16 { - count = 16 - } - for i := 0; i < count; i++ { - data := probePattern(32) - if err := wire.WriteMaskedResponse(conn, wire.StatusData, data, req.Session, wire.ModeProbe, req.Seq+uint64(i)); err != nil { - return err - } - } - return nil - default: - return wire.WriteResponse(conn, wire.StatusError, []byte("unknown probe kind")) - } - - case wire.ModeOpen: - supplied, host, port, err := parseOpen(req.Payload) - if err != nil { - return wire.WriteResponse(conn, wire.StatusError, []byte(err.Error())) - } - if !tokenEqual(supplied, token) { - return wire.WriteResponse(conn, wire.StatusError, []byte("authentication failed")) - } - if old := manager.get(req.Session); old != nil { - body := make([]byte, 4) - binary.BigEndian.PutUint32(body, uint32(maxChunk)) - return wire.WriteMaskedResponse(conn, wire.StatusOK, body, req.Session, wire.ModeOpen, req.Seq) - } - ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) - target, err := dialTarget(ctx, host, port, allowPrivate, cache, tcpBuffer) - cancel() - if err != nil { - return wire.WriteResponse(conn, wire.StatusError, []byte(err.Error())) - } - session := newStreamSession(req.Session, target, fmt.Sprintf("%s:%d", host, port), maxChunk, maxBuffer, debug) - _, created := manager.addOrGet(req.Session, session) - if created && debug != nil && debug.enabled { - debug.sessionsOpened.Add(1) - debug.activeSessions.Add(1) - debug.logf("SESSION OPEN sid=%x target=%s:%d active_sessions=%d", req.Session[:4], host, port, manager.count()) - } - body := make([]byte, 4) - binary.BigEndian.PutUint32(body, uint32(maxChunk)) - return wire.WriteMaskedResponse(conn, wire.StatusOK, body, req.Session, wire.ModeOpen, req.Seq) - - case wire.ModeUpload: - s := manager.get(req.Session) - if s == nil { - return wire.WriteResponse(conn, wire.StatusError, []byte("unknown session")) - } - if len(req.Payload) > maxChunk { - return wire.WriteResponse(conn, wire.StatusError, []byte("upload too large")) - } - if err := s.upload(req.Seq, req.Payload); err != nil { - return wire.WriteResponse(conn, wire.StatusError, []byte(err.Error())) - } - return wire.WriteResponse(conn, wire.StatusOK, nil) - - case wire.ModeDownload: - s := manager.get(req.Session) - if s == nil { - return wire.WriteResponse(conn, wire.StatusError, []byte("unknown session")) - } - if len(req.Payload) != 14 { - return wire.WriteResponse(conn, wire.StatusError, []byte("bad download request")) - } - ack := binary.BigEndian.Uint64(req.Payload[0:8]) - limit := int(binary.BigEndian.Uint32(req.Payload[8:12])) - count := int(binary.BigEndian.Uint16(req.Payload[12:14])) - if limit < 1 { - limit = 1 - } - if limit > maxChunk { - limit = maxChunk - } - if count < 1 { - count = 1 - } - if count > 256 { - count = 256 - } - s.ack(ack) - offset := req.Seq - if debug != nil && debug.enabled { - debug.pullRequests.Add(1) - } - for i := 0; i < count; i++ { - wait := time.Duration(0) - if i == 0 { - wait = pollWait - } - data, status, err := s.readAt(offset, limit, wait) - if err != nil { - return wire.WriteResponse(conn, wire.StatusError, []byte(err.Error())) - } - switch status { - case wire.StatusData: - if debug != nil && debug.enabled { - debug.dataRecords.Add(1) - } - if err := wire.WriteMaskedResponse(conn, wire.StatusData, data, req.Session, wire.ModeDownload, offset); err != nil { - return err - } - offset += uint64(len(data)) - case wire.StatusWait: - if debug != nil && debug.enabled { - debug.waitRecords.Add(1) - } - return wire.WriteResponse(conn, wire.StatusWait, nil) - case wire.StatusEOF: - return wire.WriteResponse(conn, wire.StatusEOF, nil) - default: - return wire.WriteResponse(conn, wire.StatusError, []byte("invalid session read status")) - } - } - return nil - - case wire.ModeClose: - manager.remove(req.Session) - return wire.WriteResponse(conn, wire.StatusOK, nil) - default: - return wire.WriteResponse(conn, wire.StatusError, []byte("unknown mode")) - } -} diff --git a/android/core/cmd/dragontcp-server/chunk_test.go b/android/core/cmd/dragontcp-server/chunk_test.go deleted file mode 100644 index 1801b33..0000000 --- a/android/core/cmd/dragontcp-server/chunk_test.go +++ /dev/null @@ -1,22 +0,0 @@ -package main - -import ( - "encoding/binary" - "testing" -) - -func TestParseOpenAllowsEmptyToken(t *testing.T) { - host := "example.com" - p := make([]byte, 6+len(host)) - binary.BigEndian.PutUint16(p[0:2], 0) - binary.BigEndian.PutUint16(p[2:4], uint16(len(host))) - binary.BigEndian.PutUint16(p[4:6], 443) - copy(p[6:], host) - token, gotHost, port, err := parseOpen(p) - if err != nil { - t.Fatal(err) - } - if token != "" || gotHost != host || port != 443 { - t.Fatalf("got token=%q host=%q port=%d", token, gotHost, port) - } -} diff --git a/android/core/cmd/dragontcp-server/debug.go b/android/core/cmd/dragontcp-server/debug.go deleted file mode 100644 index f23f94b..0000000 --- a/android/core/cmd/dragontcp-server/debug.go +++ /dev/null @@ -1,83 +0,0 @@ -package main - -import ( - "fmt" - "os" - "sync/atomic" - "time" -) - -type serverDebug struct { - enabled bool - chunks bool - statsEvery time.Duration - started time.Time - - sessionsOpened atomic.Uint64 - sessionsClosed atomic.Uint64 - activeSessions atomic.Int64 - bytesUp atomic.Uint64 - bytesDown atomic.Uint64 - pushRecords atomic.Uint64 - pullRequests atomic.Uint64 - dataRecords atomic.Uint64 - waitRecords atomic.Uint64 - errors atomic.Uint64 -} - -func newServerDebug(enabled, chunks bool, statsEvery time.Duration) *serverDebug { - d := &serverDebug{ - enabled: enabled || chunks, - chunks: chunks, - statsEvery: statsEvery, - started: time.Now(), - } - if d.enabled && d.statsEvery > 0 { - go d.statsLoop() - } - return d -} - -func (d *serverDebug) logf(format string, args ...any) { - if d == nil || !d.enabled { - return - } - fmt.Fprintf(os.Stderr, "%s [DEBUG] "+format+"\n", append([]any{time.Now().Format("2006-01-02 15:04:05.000")}, args...)...) -} - -func (d *serverDebug) chunkf(format string, args ...any) { - if d == nil || !d.chunks { - return - } - fmt.Fprintf(os.Stderr, "%s [CHUNK] "+format+"\n", append([]any{time.Now().Format("2006-01-02 15:04:05.000")}, args...)...) -} - -func (d *serverDebug) errorf(format string, args ...any) { - if d == nil || !d.enabled { - return - } - d.errors.Add(1) - fmt.Fprintf(os.Stderr, "%s [ERROR] "+format+"\n", append([]any{time.Now().Format("2006-01-02 15:04:05.000")}, args...)...) -} - -func (d *serverDebug) statsLoop() { - ticker := time.NewTicker(d.statsEvery) - defer ticker.Stop() - for range ticker.C { - d.logf( - "STATS uptime=%s active_connections=%d active_sessions=%d sessions_opened=%d sessions_closed=%d bytes_up=%d bytes_down=%d push_records=%d pull_requests=%d data_records=%d waits=%d errors=%d", - time.Since(d.started).Round(time.Second), - atomic.LoadInt64(&active), - d.activeSessions.Load(), - d.sessionsOpened.Load(), - d.sessionsClosed.Load(), - d.bytesUp.Load(), - d.bytesDown.Load(), - d.pushRecords.Load(), - d.pullRequests.Load(), - d.dataRecords.Load(), - d.waitRecords.Load(), - d.errors.Load(), - ) - } -} diff --git a/android/core/cmd/dragontcp-server/main.go b/android/core/cmd/dragontcp-server/main.go deleted file mode 100644 index dd33ddd..0000000 --- a/android/core/cmd/dragontcp-server/main.go +++ /dev/null @@ -1,290 +0,0 @@ -package main - -import ( - "context" - "crypto/subtle" - "flag" - "fmt" - "net" - "net/netip" - "os" - "strconv" - "strings" - "sync" - "sync/atomic" - "time" - - "dragontcp/internal/protocol" - "dragontcp/internal/wire" -) - -var active int64 - -type dnsEntry struct { - ips []netip.Addr - expires time.Time -} - -type dnsCache struct { - mu sync.RWMutex - entries map[string]dnsEntry - ttl time.Duration - max int -} - -func newDNSCache(ttl time.Duration, max int) *dnsCache { - return &dnsCache{ - entries: make(map[string]dnsEntry), - ttl: ttl, - max: max, - } -} - -func (c *dnsCache) resolve(ctx context.Context, host string) ([]netip.Addr, error) { - if ip, err := netip.ParseAddr(host); err == nil { - return []netip.Addr{ip}, nil - } - - now := time.Now() - c.mu.RLock() - entry, ok := c.entries[host] - c.mu.RUnlock() - if ok && now.Before(entry.expires) { - return entry.ips, nil - } - - ips, err := net.DefaultResolver.LookupNetIP(ctx, "ip", host) - if err != nil { - return nil, err - } - - c.mu.Lock() - if len(c.entries) >= c.max { - // Simple bounded reset keeps the hot cache cheap and prevents growth. - c.entries = make(map[string]dnsEntry, c.max) - } - c.entries[host] = dnsEntry{ips: ips, expires: now.Add(c.ttl)} - c.mu.Unlock() - - return ips, nil -} - -func tokenEqual(a, b string) bool { - if len(a) != len(b) { - return false - } - return subtle.ConstantTimeCompare([]byte(a), []byte(b)) == 1 -} - -var blockedSpecial = []netip.Prefix{ - netip.MustParsePrefix("0.0.0.0/8"), - netip.MustParsePrefix("100.64.0.0/10"), - netip.MustParsePrefix("192.0.0.0/24"), - netip.MustParsePrefix("192.0.2.0/24"), - netip.MustParsePrefix("198.18.0.0/15"), - netip.MustParsePrefix("198.51.100.0/24"), - netip.MustParsePrefix("203.0.113.0/24"), - netip.MustParsePrefix("240.0.0.0/4"), - netip.MustParsePrefix("2001:db8::/32"), -} - -func addressAllowed(addr netip.Addr, allowPrivate bool) bool { - if addr.IsUnspecified() || addr.IsMulticast() { - return false - } - - if allowPrivate { - return true - } - - if !addr.IsGlobalUnicast() || - addr.IsPrivate() || - addr.IsLoopback() || - addr.IsLinkLocalUnicast() { - return false - } - - for _, prefix := range blockedSpecial { - if prefix.Contains(addr) { - return false - } - } - - return true -} - -func dialTarget(ctx context.Context, host string, port int, allowPrivate bool, cache *dnsCache, tcpBuffer int) (net.Conn, error) { - ips, err := cache.resolve(ctx, host) - if err != nil { - return nil, err - } - - var lastErr error - var blocked []string - - d := net.Dialer{ - Timeout: 10 * time.Second, - KeepAlive: 30 * time.Second, - } - - for _, ip := range ips { - if !addressAllowed(ip, allowPrivate) { - blocked = append(blocked, ip.String()) - continue - } - - addr := net.JoinHostPort(ip.String(), strconv.Itoa(port)) - conn, err := d.DialContext(ctx, "tcp", addr) - if err == nil { - protocol.TuneTCP(conn) - protocol.TuneTCPBuffer(conn, tcpBuffer) - return conn, nil - } - lastErr = err - } - - if lastErr != nil { - return nil, lastErr - } - if len(blocked) > 0 { - return nil, fmt.Errorf("target resolves only to blocked addresses: %s", strings.Join(blocked, ",")) - } - return nil, fmt.Errorf("no usable target address") -} - -func handle( - conn net.Conn, - token string, - allowPrivate bool, - cache *dnsCache, - tcpBuffer int, - slots chan struct{}, - manager *streamManager, - chunkMax int, - bufferBytes int, - chunkPollWait time.Duration, - debug *serverDebug, -) { - defer func() { - <-slots - atomic.AddInt64(&active, -1) - _ = conn.Close() - }() - - protocol.TuneTCP(conn) - protocol.TuneTCPBuffer(conn, tcpBuffer) - - for { - _ = conn.SetDeadline(time.Now().Add(30 * time.Second)) - req, err := wire.ReadRequest(conn) - if err != nil { - return - } - if err := processWireRequest( - conn, - req, - token, - allowPrivate, - cache, - tcpBuffer, - manager, - chunkMax, - bufferBytes, - chunkPollWait, - debug, - ); err != nil { - return - } - } -} - -func main() { - var ( - host = flag.String("host", "0.0.0.0", "listen host") - port = flag.Int("port", 53, "listen port") - token = flag.String("token", "", "optional shared token") - maxConnections = flag.Int("max-connections", 20000, "max simultaneous tunnels") - allowPrivate = flag.Bool("allow-private", false, "allow private/loopback targets") - dnsCacheTTL = flag.Duration("dns-cache-ttl", 30*time.Second, "server DNS cache TTL") - dnsCacheSize = flag.Int("dns-cache-size", 4096, "maximum cached DNS hostnames") - tcpBuffer = flag.Int("tcp-buffer", 0, "optional TCP read/write buffer bytes; 0 keeps OS autotuning") - chunkMax = flag.Int("chunk-max", 1048576, "maximum adaptive chunk payload bytes (32 bytes to 1 MiB)") - chunkBuffered = flag.Int("chunk-buffered", 256, "compatibility buffer units; 256 = about 16 MiB per active session") - chunkPollWait = flag.Duration("chunk-poll-wait", 200*time.Millisecond, "server long-poll wait for chunk data") - sessionTimeout = flag.Duration("chunk-session-timeout", 2*time.Minute, "idle chunk session timeout") - debugEnabled = flag.Bool("debug", false, "log session/connect/errors and periodic statistics") - debugChunks = flag.Bool("debug-chunks", false, "log every chunk protocol record; very verbose") - debugStats = flag.Duration("debug-stats-interval", 5*time.Second, "periodic debug statistics interval; 0 disables") - ) - flag.Parse() - - if *chunkMax < 32 || *chunkMax > protocol.MaxChunkPayload { - fmt.Fprintf(os.Stderr, "--chunk-max must be between 32 and %d\n", protocol.MaxChunkPayload) - os.Exit(2) - } - if *chunkBuffered < 8 { - fmt.Fprintln(os.Stderr, "--chunk-buffered must be at least 8") - os.Exit(2) - } - - listenAddr := net.JoinHostPort(*host, strconv.Itoa(*port)) - ln, err := net.Listen("tcp", listenAddr) - if err != nil { - fmt.Fprintln(os.Stderr, err) - os.Exit(1) - } - defer ln.Close() - - fmt.Printf("DragonTCP Go server listening on %s\n", listenAddr) - fmt.Printf("max_connections=%d tcp_buffer=%d\n", *maxConnections, *tcpBuffer) - - slots := make(chan struct{}, *maxConnections) - cache := newDNSCache(*dnsCacheTTL, *dnsCacheSize) - debug := newServerDebug(*debugEnabled, *debugChunks, *debugStats) - bufferBytes := *chunkBuffered * 65536 - if bufferBytes < 1024*1024 { - bufferBytes = 1024 * 1024 - } - if bufferBytes > 64*1024*1024 { - bufferBytes = 64 * 1024 * 1024 - } - manager := newStreamManager(*sessionTimeout, debug) - fmt.Printf("binary_transport=true chunk_max=%d buffer_bytes=%d poll_wait=%s\n", *chunkMax, bufferBytes, chunkPollWait.String()) - if debug.enabled { - fmt.Printf("debug=true debug_chunks=%t stats_interval=%s\n", debug.chunks, debug.statsEvery) - } - - for { - conn, err := ln.Accept() - if err != nil { - fmt.Fprintln(os.Stderr, "accept:", err) - continue - } - - select { - case slots <- struct{}{}: - atomic.AddInt64(&active, 1) - if debug.enabled { - debug.logf("ACCEPT peer=%v active_connections=%d", conn.RemoteAddr(), atomic.LoadInt64(&active)) - } - go handle( - conn, - *token, - *allowPrivate, - cache, - *tcpBuffer, - slots, - manager, - *chunkMax, - bufferBytes, - *chunkPollWait, - debug, - ) - default: - if debug.enabled { - debug.errorf("REJECT peer=%v reason=max-connections", conn.RemoteAddr()) - } - _ = conn.Close() - } - } -} diff --git a/android/core/dragontcp-client b/android/core/dragontcp-client deleted file mode 100644 index 035b7f7..0000000 Binary files a/android/core/dragontcp-client and /dev/null differ diff --git a/android/core/dragontcp-server b/android/core/dragontcp-server deleted file mode 100644 index 07bf58e..0000000 Binary files a/android/core/dragontcp-server and /dev/null differ diff --git a/android/core/go.mod b/android/core/go.mod deleted file mode 100644 index a8d2a13..0000000 --- a/android/core/go.mod +++ /dev/null @@ -1,3 +0,0 @@ -module dragontcp - -go 1.22 diff --git a/android/core/internal/protocol/protocol.go b/android/core/internal/protocol/protocol.go deleted file mode 100644 index 555dcfd..0000000 --- a/android/core/internal/protocol/protocol.go +++ /dev/null @@ -1,211 +0,0 @@ -package protocol - -import ( - "encoding/binary" - "errors" - "fmt" - "io" - "net" - "sync" - "time" -) - -const ( - XORKey byte = 0xAD - - // MaxChunkPayload is the hard application-record payload ceiling. - // The adaptive chunk protocol may use any size from 32 bytes through 1 MiB. - MaxChunkPayload = 1024 * 1024 - - // Framed CPUSH/DATA messages include text metadata in addition to chunk - // bytes, so keep the frame ceiling comfortably above MaxChunkPayload. - MaxHandshake = 2 * 1024 * 1024 -) - -// 64 KiB balances throughput with memory use at high connection counts. -var BufferPool = sync.Pool{ - New: func() any { - b := make([]byte, 64*1024) - return &b - }, -} - -func ReadRequestFrame(r io.Reader) (uint32, uint32, []byte, error) { - var header [14]byte - if _, err := io.ReadFull(r, header[:]); err != nil { - return 0, 0, nil, err - } - - if header[0] != 'U' || header[1] != 'P' { - return 0, 0, nil, errors.New("bad request magic") - } - - requestID := binary.BigEndian.Uint32(header[2:6]) - reserved := binary.BigEndian.Uint32(header[6:10]) - length := binary.BigEndian.Uint32(header[10:14]) - - if length > MaxHandshake { - return 0, 0, nil, errors.New("handshake payload too large") - } - - payload := make([]byte, int(length)) - if _, err := io.ReadFull(r, payload); err != nil { - return 0, 0, nil, err - } - XorInPlace(payload) - - return requestID, reserved, payload, nil -} - -func WriteRequestFrame(w io.Writer, requestID uint32, payload []byte) error { - if len(payload) > MaxHandshake { - return errors.New("request frame payload too large") - } - packet := make([]byte, 14+len(payload)) - packet[0], packet[1] = 'U', 'P' - binary.BigEndian.PutUint32(packet[2:6], requestID) - binary.BigEndian.PutUint32(packet[6:10], 0) - binary.BigEndian.PutUint32(packet[10:14], uint32(len(payload))) - copy(packet[14:], payload) - XorInPlace(packet[14:]) - return writeAll(w, packet) -} - -func ReadResponseFrame(r io.Reader) (uint32, []byte, error) { - var header [10]byte - if _, err := io.ReadFull(r, header[:]); err != nil { - return 0, nil, err - } - - if header[0] != 'O' || header[1] != 'K' { - return 0, nil, fmt.Errorf("bad response magic: %q", header[:2]) - } - - requestID := binary.BigEndian.Uint32(header[2:6]) - length := binary.BigEndian.Uint32(header[6:10]) - - if length > MaxHandshake { - return 0, nil, errors.New("handshake response too large") - } - - payload := make([]byte, int(length)) - if _, err := io.ReadFull(r, payload); err != nil { - return 0, nil, err - } - XorInPlace(payload) - - return requestID, payload, nil -} - -func WriteResponseFrame(w io.Writer, requestID uint32, payload []byte) error { - if len(payload) > MaxHandshake { - return errors.New("response frame payload too large") - } - packet := make([]byte, 10+len(payload)) - packet[0], packet[1] = 'O', 'K' - binary.BigEndian.PutUint32(packet[2:6], requestID) - binary.BigEndian.PutUint32(packet[6:10], uint32(len(payload))) - copy(packet[10:], payload) - XorInPlace(packet[10:]) - return writeAll(w, packet) -} - -func writeAll(w io.Writer, b []byte) error { - for len(b) > 0 { - n, err := w.Write(b) - if err != nil { - return err - } - b = b[n:] - } - return nil -} - -func CopyXOR(dst net.Conn, src net.Conn) error { - ptr := BufferPool.Get().(*[]byte) - buf := *ptr - defer BufferPool.Put(ptr) - - for { - n, err := src.Read(buf) - if n > 0 { - chunk := buf[:n] - XorInPlace(chunk) - - if err2 := writeAll(dst, chunk); err2 != nil { - return err2 - } - - // No restore pass is needed. The next Read overwrites these bytes. - } - - if err != nil { - if errors.Is(err, io.EOF) { - return nil - } - return err - } - } -} - -func relayPair(a, b net.Conn, copier func(net.Conn, net.Conn) error) { - done := make(chan struct{}, 2) - - go func() { - _ = copier(b, a) - if cw, ok := b.(interface{ CloseWrite() error }); ok { - _ = cw.CloseWrite() - } - done <- struct{}{} - }() - - go func() { - _ = copier(a, b) - if cw, ok := a.(interface{ CloseWrite() error }); ok { - _ = cw.CloseWrite() - } - done <- struct{}{} - }() - - // Preserve normal TCP half-close semantics. The old implementation set a - // 2-second deadline on both connections after the first copy direction - // ended, which truncated slow or large responses. Wait for the remaining - // direction to drain naturally instead. - <-done - <-done -} - -func RelayXOR(a, b net.Conn) { - relayPair(a, b, CopyXOR) -} - -// RelayRaw allows Go/Linux to use the optimized TCP io.Copy path. On Linux, -// TCP-to-TCP copies can use splice, eliminating the userspace XOR/copy loop. -func RelayRaw(a, b net.Conn) { - relayPair(a, b, func(dst, src net.Conn) error { - _, err := io.Copy(dst, src) - return err - }) -} - -func TuneTCP(conn net.Conn) { - if tcp, ok := conn.(*net.TCPConn); ok { - _ = tcp.SetNoDelay(true) - _ = tcp.SetKeepAlive(true) - _ = tcp.SetKeepAlivePeriod(30 * time.Second) - } -} - -// TuneTCPBuffer optionally requests larger kernel socket buffers. A value <= 0 -// leaves Linux/Android autotuning untouched, which is the recommended default -// for large connection counts. For a small number of high-BDP mobile links, -// values such as 1048576 or 4194304 can improve throughput. -func TuneTCPBuffer(conn net.Conn, size int) { - if size <= 0 { - return - } - if tcp, ok := conn.(*net.TCPConn); ok { - _ = tcp.SetReadBuffer(size) - _ = tcp.SetWriteBuffer(size) - } -} diff --git a/android/core/internal/protocol/xor_fast32.go b/android/core/internal/protocol/xor_fast32.go deleted file mode 100644 index dff13db..0000000 --- a/android/core/internal/protocol/xor_fast32.go +++ /dev/null @@ -1,44 +0,0 @@ -//go:build arm || 386 - -package protocol - -import "unsafe" - -const xorWordMask32 uint32 = 0xADADADAD - -// XorInPlace is the 32-bit optimized path used by ARMv7/386 builds. -// It aligns once, then processes 32 bytes per iteration with native uint32 -// operations instead of a byte-at-a-time loop. -func XorInPlace(b []byte) { - n := len(b) - if n == 0 { - return - } - - i := 0 - for i < n && (uintptr(unsafe.Pointer(&b[i]))&3) != 0 { - b[i] ^= XORKey - i++ - } - - for ; i+32 <= n; i += 32 { - p := unsafe.Pointer(&b[i]) - *(*uint32)(unsafe.Add(p, 0)) ^= xorWordMask32 - *(*uint32)(unsafe.Add(p, 4)) ^= xorWordMask32 - *(*uint32)(unsafe.Add(p, 8)) ^= xorWordMask32 - *(*uint32)(unsafe.Add(p, 12)) ^= xorWordMask32 - *(*uint32)(unsafe.Add(p, 16)) ^= xorWordMask32 - *(*uint32)(unsafe.Add(p, 20)) ^= xorWordMask32 - *(*uint32)(unsafe.Add(p, 24)) ^= xorWordMask32 - *(*uint32)(unsafe.Add(p, 28)) ^= xorWordMask32 - } - - for ; i+4 <= n; i += 4 { - p := (*uint32)(unsafe.Pointer(&b[i])) - *p ^= xorWordMask32 - } - - for ; i < n; i++ { - b[i] ^= XORKey - } -} diff --git a/android/core/internal/protocol/xor_fast64.go b/android/core/internal/protocol/xor_fast64.go deleted file mode 100644 index 9faa342..0000000 --- a/android/core/internal/protocol/xor_fast64.go +++ /dev/null @@ -1,50 +0,0 @@ -//go:build amd64 || arm64 - -package protocol - -import "unsafe" - -const xorWordMask uint64 = 0xADADADADADADADAD - -// XorInPlace is optimized for 64-bit targets (amd64/arm64). -// -// It aligns the input once, then XORs 64 bytes per loop iteration using -// eight native 64-bit operations. This removes the encoding/binary call -// overhead from the hot relay path and lets the compiler generate a tight -// load/xor/store loop. -func XorInPlace(b []byte) { - n := len(b) - if n == 0 { - return - } - - i := 0 - - // Align the pointer for native uint64 accesses. This is normally already - // aligned for pooled relay buffers, but also makes this safe for subslices. - for i < n && (uintptr(unsafe.Pointer(&b[i]))&7) != 0 { - b[i] ^= XORKey - i++ - } - - for ; i+64 <= n; i += 64 { - p := unsafe.Pointer(&b[i]) - *(*uint64)(unsafe.Add(p, 0)) ^= xorWordMask - *(*uint64)(unsafe.Add(p, 8)) ^= xorWordMask - *(*uint64)(unsafe.Add(p, 16)) ^= xorWordMask - *(*uint64)(unsafe.Add(p, 24)) ^= xorWordMask - *(*uint64)(unsafe.Add(p, 32)) ^= xorWordMask - *(*uint64)(unsafe.Add(p, 40)) ^= xorWordMask - *(*uint64)(unsafe.Add(p, 48)) ^= xorWordMask - *(*uint64)(unsafe.Add(p, 56)) ^= xorWordMask - } - - for ; i+8 <= n; i += 8 { - p := (*uint64)(unsafe.Pointer(&b[i])) - *p ^= xorWordMask - } - - for ; i < n; i++ { - b[i] ^= XORKey - } -} diff --git a/android/core/internal/protocol/xor_generic.go b/android/core/internal/protocol/xor_generic.go deleted file mode 100644 index 5a238ea..0000000 --- a/android/core/internal/protocol/xor_generic.go +++ /dev/null @@ -1,10 +0,0 @@ -//go:build !amd64 && !arm64 && !arm && !386 - -package protocol - -// Generic fallback for 32-bit and uncommon architectures. -func XorInPlace(b []byte) { - for i := range b { - b[i] ^= XORKey - } -} diff --git a/android/core/internal/wire/protocol.go b/android/core/internal/wire/protocol.go deleted file mode 100644 index 7e8730c..0000000 --- a/android/core/internal/wire/protocol.go +++ /dev/null @@ -1,177 +0,0 @@ -package wire - -import ( - "crypto/sha256" - "encoding/binary" - "errors" - "fmt" - "io" -) - -const ( - RequestHeaderSize = 29 - ResponseHeaderSize = 5 - MaxPayload = 2 * 1024 * 1024 - - ModeProbe byte = 0 - ModeOpen byte = 1 - ModeUpload byte = 2 - ModeDownload byte = 3 - ModeClose byte = 4 - - StatusOK byte = 0 - StatusError byte = 1 - StatusData byte = 2 - StatusWait byte = 3 - StatusEOF byte = 4 - - ProbeUpload byte = 1 - ProbeDownload byte = 2 - ProbeKeepalive byte = 3 - ProbeBatch byte = 4 -) - -var ProbeMagic = [4]byte{'D', 'T', 'P', '2'} - -type SessionID [16]byte - -type Request struct { - Mode byte - Session SessionID - Seq uint64 - Payload []byte -} - -func MaskInPlace(data []byte, sid SessionID, mode byte, seq uint64, response bool) { - if len(data) == 0 { - return - } - - var seed [30]byte - copy(seed[:16], sid[:]) - seed[16] = mode - binary.BigEndian.PutUint64(seed[17:25], seq) - if response { - seed[25] = 1 - } - - var counter uint32 - for off := 0; off < len(data); { - binary.BigEndian.PutUint32(seed[26:30], counter) - block := sha256.Sum256(seed[:]) - n := len(data) - off - if n > len(block) { - n = len(block) - } - for i := 0; i < n; i++ { - data[off+i] ^= block[i] - } - off += n - counter++ - } -} - -func WriteRequest(w io.Writer, mode byte, sid SessionID, seq uint64, plaintext []byte) error { - if len(plaintext) > MaxPayload { - return fmt.Errorf("request payload too large: %d", len(plaintext)) - } - - packet := make([]byte, RequestHeaderSize+len(plaintext)) - packet[0] = mode - copy(packet[1:17], sid[:]) - binary.BigEndian.PutUint64(packet[17:25], seq) - binary.BigEndian.PutUint32(packet[25:29], uint32(len(plaintext))) - copy(packet[29:], plaintext) - MaskInPlace(packet[29:], sid, mode, seq, false) - return writeAll(w, packet) -} - -func ReadRequest(r io.Reader) (Request, error) { - var req Request - var header [RequestHeaderSize]byte - if _, err := io.ReadFull(r, header[:]); err != nil { - return req, err - } - - req.Mode = header[0] - copy(req.Session[:], header[1:17]) - req.Seq = binary.BigEndian.Uint64(header[17:25]) - n := binary.BigEndian.Uint32(header[25:29]) - if n > MaxPayload { - return req, errors.New("request payload too large") - } - - if n > 0 { - req.Payload = make([]byte, int(n)) - if _, err := io.ReadFull(r, req.Payload); err != nil { - return req, err - } - MaskInPlace(req.Payload, req.Session, req.Mode, req.Seq, false) - } - return req, nil -} - -func WriteResponse(w io.Writer, status byte, body []byte) error { - if len(body) > MaxPayload { - return fmt.Errorf("response body too large: %d", len(body)) - } - packet := make([]byte, ResponseHeaderSize+len(body)) - packet[0] = status - binary.BigEndian.PutUint32(packet[1:5], uint32(len(body))) - copy(packet[5:], body) - return writeAll(w, packet) -} - -func WriteMaskedResponse(w io.Writer, status byte, body []byte, sid SessionID, mode byte, seq uint64) error { - if len(body) > MaxPayload { - return fmt.Errorf("response body too large: %d", len(body)) - } - packet := make([]byte, ResponseHeaderSize+len(body)) - packet[0] = status - binary.BigEndian.PutUint32(packet[1:5], uint32(len(body))) - copy(packet[5:], body) - MaskInPlace(packet[5:], sid, mode, seq, true) - return writeAll(w, packet) -} - -func ReadResponse(r io.Reader) (byte, []byte, error) { - var header [ResponseHeaderSize]byte - if _, err := io.ReadFull(r, header[:]); err != nil { - return 0, nil, err - } - n := binary.BigEndian.Uint32(header[1:5]) - if n > MaxPayload { - return 0, nil, errors.New("response body too large") - } - var body []byte - if n > 0 { - body = make([]byte, int(n)) - if _, err := io.ReadFull(r, body); err != nil { - return 0, nil, err - } - } - return header[0], body, nil -} - -func DecodeMaskedResponse(status byte, body []byte, sid SessionID, mode byte, seq uint64) []byte { - if len(body) == 0 || status == StatusError { - return body - } - out := append([]byte(nil), body...) - MaskInPlace(out, sid, mode, seq, true) - return out -} - -func writeAll(w io.Writer, b []byte) error { - for len(b) > 0 { - n, err := w.Write(b) - if err != nil { - return err - } - if n <= 0 { - return io.ErrShortWrite - } - b = b[n:] - } - return nil -} diff --git a/android/core/internal/wire/protocol_test.go b/android/core/internal/wire/protocol_test.go deleted file mode 100644 index 8302467..0000000 --- a/android/core/internal/wire/protocol_test.go +++ /dev/null @@ -1,29 +0,0 @@ -package wire - -import ( - "bytes" - "testing" -) - -func TestMaskChangesWithSequenceAndRoundTrips(t *testing.T) { - var sid SessionID - for i := range sid { sid[i] = byte(i+1) } - plain := bytes.Repeat([]byte("DragonTCP"), 100) - a := append([]byte(nil), plain...) - b := append([]byte(nil), plain...) - MaskInPlace(a, sid, ModeUpload, 1, false) - MaskInPlace(b, sid, ModeUpload, 2, false) - if bytes.Equal(a, b) { t.Fatal("different sequences produced identical wire bytes") } - MaskInPlace(a, sid, ModeUpload, 1, false) - if !bytes.Equal(a, plain) { t.Fatal("mask did not round-trip") } -} - -func BenchmarkMask1MiB(b *testing.B) { - var sid SessionID - data := make([]byte, 1024*1024) - b.SetBytes(int64(len(data))) - b.ResetTimer() - for i:=0;i { try (BufferedReader br = new BufferedReader(new InputStreamReader(process.getInputStream()))) { diff --git a/android/android/src/com/dragontcp/client/LogActivity.java b/android/src/com/dragontcp/client/LogActivity.java similarity index 100% rename from android/android/src/com/dragontcp/client/LogActivity.java rename to android/src/com/dragontcp/client/LogActivity.java diff --git a/android/android/src/com/dragontcp/client/MainActivity.java b/android/src/com/dragontcp/client/MainActivity.java similarity index 77% rename from android/android/src/com/dragontcp/client/MainActivity.java rename to android/src/com/dragontcp/client/MainActivity.java index f8d305f..6f3fefe 100644 --- a/android/android/src/com/dragontcp/client/MainActivity.java +++ b/android/src/com/dragontcp/client/MainActivity.java @@ -12,7 +12,9 @@ import android.graphics.drawable.GradientDrawable; import android.net.VpnService; import android.os.Build; import android.os.Bundle; +import android.text.Editable; import android.text.InputType; +import android.text.TextWatcher; import android.view.Gravity; import android.view.View; import android.view.ViewGroup; @@ -28,6 +30,8 @@ public class MainActivity extends Activity { private static final int VPN_REQUEST = 100; private static final String PREFS = "dragontcp"; + private static final int BATCH_LIMIT = 256; + private static final int BG = Color.rgb(11, 15, 20); private static final int CARD = Color.rgb(22, 28, 36); private static final int FIELD = Color.rgb(14, 19, 26); @@ -39,15 +43,19 @@ public class MainActivity extends Activity { private static final int DISABLED = Color.rgb(48, 56, 68); private static final int OK = Color.rgb(82, 201, 143); private static final int WARN = Color.rgb(240, 177, 83); + private static final int BAD = Color.rgb(232, 116, 116); private EditText server; private EditText port; private EditText token; private EditText chunkMax; private EditText chunkMin; + private EditText batchMax; + private EditText batchMin; private EditText reconnect; private EditText timeout; + private TextView batchHint; private Button connectButton; private Button stopButton; private Button logsButton; @@ -73,6 +81,7 @@ public class MainActivity extends Activity { forceDarkSystemUi(); buildUi(); loadSettings(); + updateBatchHint(); } @Override @@ -141,6 +150,7 @@ public class MainActivity extends Activity { ViewGroup.LayoutParams.WRAP_CONTENT )); + // ---------------------------------------------------------- connection LinearLayout connectionCard = card("CONNECTION"); server = addField(connectionCard, "Server", "Server IP or hostname", "", false, false); LinearLayout connectionRow = row(); @@ -149,22 +159,54 @@ public class MainActivity extends Activity { connectionCard.addView(connectionRow); settings.addView(connectionCard, cardParams()); - LinearLayout transportCard = card("TRANSPORT"); + // --------------------------------------------------------- record size + LinearLayout sizeCard = card("RECORD SIZE (BYTES)"); LinearLayout chunks = row(); chunkMax = addFieldToRow(chunks, "Max chunk", "1048576", "1048576", true, false, 0.58f); chunkMin = addFieldToRow(chunks, "Min chunk", "32", "32", true, false, 0.42f); - transportCard.addView(chunks); + sizeCard.addView(chunks); + sizeCard.addView(hint("Probed automatically on connect, then adapted if the path changes.")); + settings.addView(sizeCard, cardParams()); + // ------------------------------------------------------- download batch + LinearLayout batchCard = card("DOWNLOAD BATCH"); + batchCard.addView(hint( + "How many records one download request may return. This is a transport " + + "setting, not a thread count." + )); + LinearLayout batch = row(); + batchMax = addFieldToRow(batch, "Batch max", "1", "1", true, false, 0.5f); + batchMin = addFieldToRow(batch, "Batch min", "1", "1", true, false, 0.5f); + batchCard.addView(batch); + + batchHint = text("", 11, MUTED, true); + batchHint.setLineSpacing(0, 1.08f); + batchHint.setPadding(dp(2), dp(2), dp(2), dp(2)); + batchCard.addView(batchHint); + batchCard.addView(hint( + "Set both to the same number to pin the batch: the depth never grows or " + + "shrinks, which is what paths that only work at one specific size need." + )); + settings.addView(batchCard, cardParams()); + + TextWatcher batchWatcher = new TextWatcher() { + @Override public void beforeTextChanged(CharSequence s, int a, int b, int c) {} + @Override public void onTextChanged(CharSequence s, int a, int b, int c) {} + @Override public void afterTextChanged(Editable s) { updateBatchHint(); } + }; + batchMax.addTextChangedListener(batchWatcher); + batchMin.addTextChangedListener(batchWatcher); + + // ------------------------------------------------------------ advanced + LinearLayout advancedCard = card("ADVANCED"); LinearLayout timing = row(); reconnect = addFieldToRow(timing, "Reconnect every", "0 = persistent", "0", true, false, 0.58f); timeout = addFieldToRow(timing, "Timeout (s)", "2", "2", true, false, 0.42f); - transportCard.addView(timing); - - TextView fixed = text("Auto path probe • 0 reconnect = persistent", 11, MUTED, false); - fixed.setPadding(dp(2), dp(6), dp(2), dp(2)); - transportCard.addView(fixed); - settings.addView(transportCard, cardParams()); + advancedCard.addView(timing); + advancedCard.addView(hint("0 reconnect = persistent • 1 = auto • N = rotate every N requests")); + settings.addView(advancedCard, cardParams()); + // ------------------------------------------------------------- buttons LinearLayout buttons = row(); buttons.setPadding(0, dp(4), 0, dp(8)); connectButton = actionButton("CONNECT"); @@ -207,6 +249,49 @@ public class MainActivity extends Activity { updateConnectionUi(false, "DISCONNECTED"); } + /** Describes the batch configuration in words, live, as the user types. */ + private void updateBatchHint() { + if (batchHint == null) return; + Integer max = readInt(batchMax); + Integer min = readInt(batchMin); + + if (max == null || min == null) { + batchHint.setTextColor(MUTED); + batchHint.setText("Enter a value from 1 to " + BATCH_LIMIT + "."); + return; + } + if (max < 1 || max > BATCH_LIMIT || min < 1 || min > BATCH_LIMIT) { + batchHint.setTextColor(BAD); + batchHint.setText("Batch values must be 1-" + BATCH_LIMIT + "."); + return; + } + if (min > max) { + batchHint.setTextColor(BAD); + batchHint.setText("Batch min must not be greater than batch max."); + return; + } + if (min == max) { + batchHint.setTextColor(OK); + if (max == 1) { + batchHint.setText("Fixed: 1 record per request, never adapts."); + } else { + batchHint.setText("Pinned: exactly " + max + " records per request, never adapts."); + } + return; + } + batchHint.setTextColor(ACCENT); + batchHint.setText("Adaptive: starts at " + max + ", falls back toward " + min + + " on errors, recovers to " + max + "."); + } + + private Integer readInt(EditText field) { + if (field == null) return null; + String raw = field.getText().toString().trim(); + if (raw.isEmpty()) return null; + try { return Integer.valueOf(Integer.parseInt(raw)); } + catch (Exception e) { return null; } + } + private LinearLayout card(String title) { LinearLayout card = new LinearLayout(this); card.setOrientation(LinearLayout.VERTICAL); @@ -219,6 +304,13 @@ public class MainActivity extends Activity { return card; } + private TextView hint(String value) { + TextView t = text(value, 11, MUTED, false); + t.setLineSpacing(0, 1.08f); + t.setPadding(dp(2), dp(2), dp(2), dp(6)); + return t; + } + private LinearLayout.LayoutParams cardParams() { LinearLayout.LayoutParams p = new LinearLayout.LayoutParams( ViewGroup.LayoutParams.MATCH_PARENT, @@ -356,6 +448,8 @@ public class MainActivity extends Activity { i.putExtra(DragonService.EXTRA_TOKEN, p.getString("token", "")); i.putExtra(DragonService.EXTRA_CHUNK_MAX, p.getInt("max", 1048576)); i.putExtra(DragonService.EXTRA_CHUNK_MIN, p.getInt("min", 32)); + i.putExtra(DragonService.EXTRA_BATCH_MAX, p.getInt("batchMax", 1)); + i.putExtra(DragonService.EXTRA_BATCH_MIN, p.getInt("batchMin", 1)); i.putExtra(DragonService.EXTRA_RECONNECT, p.getInt("reconnect", 0)); i.putExtra(DragonService.EXTRA_TIMEOUT, p.getInt("timeout", 2)); if (Build.VERSION.SDK_INT >= 26) startForegroundService(i); else startService(i); @@ -367,6 +461,9 @@ public class MainActivity extends Activity { int p = parse(port, 1, 65535, "Port"); int max = parse(chunkMax, 32, 1048576, "Max chunk"); int min = parse(chunkMin, 32, max, "Min chunk"); + int bMax = parse(batchMax, 1, BATCH_LIMIT, "Batch max"); + int bMin = parse(batchMin, 1, BATCH_LIMIT, "Batch min"); + if (bMin > bMax) throw new IllegalArgumentException("Batch min must not exceed batch max"); int rec = parse(reconnect, 0, 1000000, "Reconnect every"); int tout = parse(timeout, 1, 120, "Timeout"); @@ -376,6 +473,8 @@ public class MainActivity extends Activity { .putString("token", token.getText().toString()) .putInt("max", max) .putInt("min", min) + .putInt("batchMax", bMax) + .putInt("batchMin", bMin) .putInt("reconnect", rec) .putInt("timeout", tout) .apply(); @@ -396,6 +495,8 @@ public class MainActivity extends Activity { token.setText(p.getString("token", "")); chunkMax.setText(Integer.toString(p.getInt("max", 1048576))); chunkMin.setText(Integer.toString(p.getInt("min", 32))); + batchMax.setText(Integer.toString(p.getInt("batchMax", 1))); + batchMin.setText(Integer.toString(p.getInt("batchMin", 1))); reconnect.setText(Integer.toString(p.getInt("reconnect", 0))); timeout.setText(Integer.toString(p.getInt("timeout", 2))); } diff --git a/android/android/src/tech/xvanturing/freeproxy/data/model/ProxyProfile.kt b/android/src/tech/xvanturing/freeproxy/data/model/ProxyProfile.kt similarity index 100% rename from android/android/src/tech/xvanturing/freeproxy/data/model/ProxyProfile.kt rename to android/src/tech/xvanturing/freeproxy/data/model/ProxyProfile.kt diff --git a/android/android/src/tech/xvanturing/freeproxy/vpn/AppResolver.kt b/android/src/tech/xvanturing/freeproxy/vpn/AppResolver.kt similarity index 100% rename from android/android/src/tech/xvanturing/freeproxy/vpn/AppResolver.kt rename to android/src/tech/xvanturing/freeproxy/vpn/AppResolver.kt diff --git a/android/android/src/tech/xvanturing/freeproxy/vpn/TcpSession.kt b/android/src/tech/xvanturing/freeproxy/vpn/TcpSession.kt similarity index 100% rename from android/android/src/tech/xvanturing/freeproxy/vpn/TcpSession.kt rename to android/src/tech/xvanturing/freeproxy/vpn/TcpSession.kt diff --git a/android/android/src/tech/xvanturing/freeproxy/vpn/TunWriter.kt b/android/src/tech/xvanturing/freeproxy/vpn/TunWriter.kt similarity index 100% rename from android/android/src/tech/xvanturing/freeproxy/vpn/TunWriter.kt rename to android/src/tech/xvanturing/freeproxy/vpn/TunWriter.kt diff --git a/android/android/src/tech/xvanturing/freeproxy/vpn/TunnelEngine.kt b/android/src/tech/xvanturing/freeproxy/vpn/TunnelEngine.kt similarity index 100% rename from android/android/src/tech/xvanturing/freeproxy/vpn/TunnelEngine.kt rename to android/src/tech/xvanturing/freeproxy/vpn/TunnelEngine.kt diff --git a/android/android/src/tech/xvanturing/freeproxy/vpn/UdpSession.kt b/android/src/tech/xvanturing/freeproxy/vpn/UdpSession.kt similarity index 100% rename from android/android/src/tech/xvanturing/freeproxy/vpn/UdpSession.kt rename to android/src/tech/xvanturing/freeproxy/vpn/UdpSession.kt diff --git a/android/android/src/tech/xvanturing/freeproxy/vpn/VpnStateHolder.kt b/android/src/tech/xvanturing/freeproxy/vpn/VpnStateHolder.kt similarity index 100% rename from android/android/src/tech/xvanturing/freeproxy/vpn/VpnStateHolder.kt rename to android/src/tech/xvanturing/freeproxy/vpn/VpnStateHolder.kt diff --git a/android/android/src/tech/xvanturing/freeproxy/vpn/dns/DnsBlocker.kt b/android/src/tech/xvanturing/freeproxy/vpn/dns/DnsBlocker.kt similarity index 100% rename from android/android/src/tech/xvanturing/freeproxy/vpn/dns/DnsBlocker.kt rename to android/src/tech/xvanturing/freeproxy/vpn/dns/DnsBlocker.kt diff --git a/android/android/src/tech/xvanturing/freeproxy/vpn/log/TunnelLog.kt b/android/src/tech/xvanturing/freeproxy/vpn/log/TunnelLog.kt similarity index 100% rename from android/android/src/tech/xvanturing/freeproxy/vpn/log/TunnelLog.kt rename to android/src/tech/xvanturing/freeproxy/vpn/log/TunnelLog.kt diff --git a/android/android/src/tech/xvanturing/freeproxy/vpn/net/ByteCodec.kt b/android/src/tech/xvanturing/freeproxy/vpn/net/ByteCodec.kt similarity index 100% rename from android/android/src/tech/xvanturing/freeproxy/vpn/net/ByteCodec.kt rename to android/src/tech/xvanturing/freeproxy/vpn/net/ByteCodec.kt diff --git a/android/android/src/tech/xvanturing/freeproxy/vpn/net/Checksum.kt b/android/src/tech/xvanturing/freeproxy/vpn/net/Checksum.kt similarity index 100% rename from android/android/src/tech/xvanturing/freeproxy/vpn/net/Checksum.kt rename to android/src/tech/xvanturing/freeproxy/vpn/net/Checksum.kt diff --git a/android/android/src/tech/xvanturing/freeproxy/vpn/net/DnsMessage.kt b/android/src/tech/xvanturing/freeproxy/vpn/net/DnsMessage.kt similarity index 100% rename from android/android/src/tech/xvanturing/freeproxy/vpn/net/DnsMessage.kt rename to android/src/tech/xvanturing/freeproxy/vpn/net/DnsMessage.kt diff --git a/android/android/src/tech/xvanturing/freeproxy/vpn/net/DnsResponse.kt b/android/src/tech/xvanturing/freeproxy/vpn/net/DnsResponse.kt similarity index 100% rename from android/android/src/tech/xvanturing/freeproxy/vpn/net/DnsResponse.kt rename to android/src/tech/xvanturing/freeproxy/vpn/net/DnsResponse.kt diff --git a/android/android/src/tech/xvanturing/freeproxy/vpn/net/HostRegistry.kt b/android/src/tech/xvanturing/freeproxy/vpn/net/HostRegistry.kt similarity index 100% rename from android/android/src/tech/xvanturing/freeproxy/vpn/net/HostRegistry.kt rename to android/src/tech/xvanturing/freeproxy/vpn/net/HostRegistry.kt diff --git a/android/android/src/tech/xvanturing/freeproxy/vpn/net/IpHeaders.kt b/android/src/tech/xvanturing/freeproxy/vpn/net/IpHeaders.kt similarity index 100% rename from android/android/src/tech/xvanturing/freeproxy/vpn/net/IpHeaders.kt rename to android/src/tech/xvanturing/freeproxy/vpn/net/IpHeaders.kt diff --git a/android/android/src/tech/xvanturing/freeproxy/vpn/net/PacketBuilder.kt b/android/src/tech/xvanturing/freeproxy/vpn/net/PacketBuilder.kt similarity index 100% rename from android/android/src/tech/xvanturing/freeproxy/vpn/net/PacketBuilder.kt rename to android/src/tech/xvanturing/freeproxy/vpn/net/PacketBuilder.kt diff --git a/android/android/src/tech/xvanturing/freeproxy/vpn/net/SessionKey.kt b/android/src/tech/xvanturing/freeproxy/vpn/net/SessionKey.kt similarity index 100% rename from android/android/src/tech/xvanturing/freeproxy/vpn/net/SessionKey.kt rename to android/src/tech/xvanturing/freeproxy/vpn/net/SessionKey.kt diff --git a/android/android/src/tech/xvanturing/freeproxy/vpn/proxy/ProxyClient.kt b/android/src/tech/xvanturing/freeproxy/vpn/proxy/ProxyClient.kt similarity index 100% rename from android/android/src/tech/xvanturing/freeproxy/vpn/proxy/ProxyClient.kt rename to android/src/tech/xvanturing/freeproxy/vpn/proxy/ProxyClient.kt diff --git a/android/android/src/tech/xvanturing/freeproxy/vpn/proxy/SocketProtector.kt b/android/src/tech/xvanturing/freeproxy/vpn/proxy/SocketProtector.kt similarity index 100% rename from android/android/src/tech/xvanturing/freeproxy/vpn/proxy/SocketProtector.kt rename to android/src/tech/xvanturing/freeproxy/vpn/proxy/SocketProtector.kt diff --git a/bin/dragontcp-hybrid-client-linux-amd64 b/bin/dragontcp-hybrid-client-linux-amd64 index cee6698..f7223aa 100644 Binary files a/bin/dragontcp-hybrid-client-linux-amd64 and b/bin/dragontcp-hybrid-client-linux-amd64 differ diff --git a/bin/dragontcp-hybrid-server-linux-amd64 b/bin/dragontcp-hybrid-server-linux-amd64 index 59abf36..f44b5f4 100644 Binary files a/bin/dragontcp-hybrid-server-linux-amd64 and b/bin/dragontcp-hybrid-server-linux-amd64 differ diff --git a/bin/dragontcp-hybrid-server-linux-arm64 b/bin/dragontcp-hybrid-server-linux-arm64 index 1bccd9b..7e55ea5 100644 Binary files a/bin/dragontcp-hybrid-server-linux-arm64 and b/bin/dragontcp-hybrid-server-linux-arm64 differ diff --git a/build_core.ps1 b/build_core.ps1 index dfad4c9..b1cec36 100644 --- a/build_core.ps1 +++ b/build_core.ps1 @@ -15,12 +15,17 @@ param( # Only build the Android client .so (skip the Linux servers). [switch]$ClientOnly, + # Where to drop libdragontcp_client.so. Defaults to android\lib\arm64-v8a + # under this script; build_apk.ps1 passes its own app directory explicitly, + # which is what makes this work regardless of how the tree is nested. + [string]$AndroidLibDir, # Path to the go executable. [string]$GoBin = $(if ($env:GO_BIN) { $env:GO_BIN } else { 'go' }) ) $ErrorActionPreference = 'Stop' $Root = $PSScriptRoot +if (-not $AndroidLibDir) { $AndroidLibDir = Join-Path $Root 'android\lib\arm64-v8a' } function Invoke-Go { param([hashtable]$Env, [string[]]$GoArgs) @@ -40,16 +45,20 @@ function Invoke-Go { $go = Get-Command $GoBin -ErrorAction SilentlyContinue if (-not $go) { throw "Go compiler not found. Install from https://go.dev/dl/ or set -GoBin." } -New-Item -ItemType Directory -Force -Path "$Root\bin", "$Root\android\lib\arm64-v8a" | Out-Null +New-Item -ItemType Directory -Force -Path "$Root\bin", $AndroidLibDir | Out-Null Push-Location "$Root\core" try { $ldflags = '-ldflags=-s -w' Write-Host '[core] Android ARM64 client...' -ForegroundColor Cyan Invoke-Go @{ CGO_ENABLED = '0'; GOOS = 'android'; GOARCH = 'arm64' } ` - @('build', '-trimpath', $ldflags, '-o', "$Root\android\lib\arm64-v8a\libdragontcp_client.so", './cmd/dragontcp-client') + @('build', '-trimpath', $ldflags, '-o', (Join-Path $AndroidLibDir 'libdragontcp_client.so'), './cmd/dragontcp-client') if (-not $ClientOnly) { + Write-Host '[core] Linux AMD64 client...' -ForegroundColor Cyan + Invoke-Go @{ CGO_ENABLED = '0'; GOOS = 'linux'; GOARCH = 'amd64' } ` + @('build', '-trimpath', $ldflags, '-o', "$Root\bin\dragontcp-hybrid-client-linux-amd64", './cmd/dragontcp-client') + Write-Host '[core] Linux AMD64 server...' -ForegroundColor Cyan Invoke-Go @{ CGO_ENABLED = '0'; GOOS = 'linux'; GOARCH = 'amd64' } ` @('build', '-trimpath', $ldflags, '-o', "$Root\bin\dragontcp-hybrid-server-linux-amd64", './cmd/dragontcp-server') diff --git a/core/cmd/dragontcp-client/chunk.go b/core/cmd/dragontcp-client/chunk.go index 6649748..5efe873 100644 --- a/core/cmd/dragontcp-client/chunk.go +++ b/core/cmd/dragontcp-client/chunk.go @@ -27,6 +27,7 @@ type chunkClientOptions struct { pollDelay time.Duration txnTimeout time.Duration tcpBuffer int + minPipeline int maxPipeline int } @@ -526,6 +527,7 @@ type chunkConn struct { consumedOffset uint64 eof bool pipeline int + minPipeline int maxPipeline int closeOnce sync.Once @@ -556,6 +558,12 @@ func openChunkTunnel(serverAddr, token, targetHost string, targetPort int, opts if opts.maxPipeline > 256 { opts.maxPipeline = 256 } + if opts.minPipeline < 1 { + opts.minPipeline = 1 + } + if opts.minPipeline > opts.maxPipeline { + opts.minPipeline = opts.maxPipeline + } profile := getPathProfile(serverAddr, token, opts) reconnect := opts.reconnectEvery @@ -620,8 +628,11 @@ func openChunkTunnel(serverAddr, token, targetHost string, targetPort int, opts downloadLane: newRequestLane(serverAddr, opts.tcpBuffer, reconnect, opts.txnTimeout), // Start at the user-configured ceiling. On transport failures the // pipeline is halved; successful data responses grow it back by one, - // always staying inside 1..maxPipeline. A ceiling of 1 is fixed. + // always staying inside minPipeline..maxPipeline. When the two bounds + // are equal the depth is pinned and never adapts, which is what paths + // that only work at one specific batch size need. pipeline: opts.maxPipeline, + minPipeline: opts.minPipeline, maxPipeline: opts.maxPipeline, } c.upSizer = newAdaptiveSizer("upload", upStart, opts) @@ -629,6 +640,53 @@ func openChunkTunnel(serverAddr, token, targetHost string, targetPort int, opts return c, nil } +// pinnedBatch reports whether the batch depth is fixed. A pinned depth never +// grows or shrinks: some paths only deliver correctly at one specific number of +// records per request, so the adaptive controller must stay out of the way. +func (c *chunkConn) pinnedBatch() bool { return c.minPipeline >= c.maxPipeline } + +// batchCount is how many records the next download request will ask for. +func (c *chunkConn) batchCount(chunk int) int { + count := c.pipeline + if count < c.minPipeline { + count = c.minPipeline + } + if count > c.maxPipeline { + count = c.maxPipeline + } + // Bound each batch to roughly 1 MiB of useful data, but never below the + // configured floor: a pinned depth is a path requirement, not a hint. + if maxCount := (1024 * 1024) / maxInt(chunk, 1); maxCount < count { + count = maxInt(maxCount, c.minPipeline) + } + return count +} + +// growPipeline widens the batch by one after a successful data response. +func (c *chunkConn) growPipeline() { + if c.pinnedBatch() { + return + } + if c.pipeline < c.maxPipeline { + c.pipeline++ + } +} + +// shrinkPipeline halves the batch after a transport failure. It reports the old +// and new depth, and whether anything actually changed; when it returns false +// the caller should shrink the record size instead. +func (c *chunkConn) shrinkPipeline() (int, int, bool) { + if c.pinnedBatch() || c.pipeline <= c.minPipeline { + return c.pipeline, c.pipeline, false + } + old := c.pipeline + c.pipeline /= 2 + if c.pipeline < c.minPipeline { + c.pipeline = c.minPipeline + } + return old, c.pipeline, old != c.pipeline +} + func (c *chunkConn) fillReadBuffer() error { if c.eof { return io.EOF @@ -636,17 +694,7 @@ func (c *chunkConn) fillReadBuffer() error { minFailures := 0 for len(c.readBuf) == 0 && !c.eof { chunk := c.downSizer.Current() - count := c.pipeline - if count < 1 { - count = 1 - } - if count > c.maxPipeline { - count = c.maxPipeline - } - // Bound each batch to roughly 1 MiB of useful data. - if maxCount := (1024 * 1024) / maxInt(chunk, 1); maxCount < count { - count = maxInt(maxCount, 1) - } + count := c.batchCount(chunk) data, status, err := c.downloadLane.download(c.sid, c.downloadOffset, c.consumedOffset, chunk, count) for _, part := range data { @@ -655,20 +703,16 @@ func (c *chunkConn) fillReadBuffer() error { } if len(data) > 0 { c.downSizer.Success(chunk) - if c.pipeline < c.maxPipeline { - c.pipeline++ - } + c.growPipeline() minFailures = 0 } if err != nil { - if c.pipeline > 1 { - old := c.pipeline - c.pipeline /= 2 - if c.pipeline < 1 { - c.pipeline = 1 - } - if c.opts.adaptLog && old != c.pipeline { - fmt.Printf("adaptive download pipeline: %d -> %d after transport failure\n", old, c.pipeline) + // Shrink the batch first, then the record size. When the batch is + // pinned (min == max) the depth is left alone entirely and only the + // record size adapts. + if old, next, shrank := c.shrinkPipeline(); shrank { + if c.opts.adaptLog { + fmt.Printf("adaptive download batch: %d -> %d after transport failure\n", old, next) } } else { old, next := c.downSizer.Failure(chunk) diff --git a/core/cmd/dragontcp-client/chunk_test.go b/core/cmd/dragontcp-client/chunk_test.go index 16f0a24..af1470c 100644 --- a/core/cmd/dragontcp-client/chunk_test.go +++ b/core/cmd/dragontcp-client/chunk_test.go @@ -23,6 +23,76 @@ func TestAdaptiveSizerRecoversFromMinimum(t *testing.T) { } } +func newTestConn(min, max int) *chunkConn { + return &chunkConn{pipeline: max, minPipeline: min, maxPipeline: max} +} + +func TestPinnedBatchNeverAdapts(t *testing.T) { + c := newTestConn(5, 5) + if got := c.batchCount(1400); got != 5 { + t.Fatalf("pinned batch should request 5 records, got %d", got) + } + for i := 0; i < 10; i++ { + if _, _, shrank := c.shrinkPipeline(); shrank { + t.Fatal("pinned batch shrank on transport failure") + } + c.growPipeline() + } + if c.pipeline != 5 { + t.Fatalf("pinned batch drifted to %d", c.pipeline) + } + if got := c.batchCount(1400); got != 5 { + t.Fatalf("pinned batch should still request 5 records, got %d", got) + } +} + +func TestPinnedBatchSurvivesOneMiBCap(t *testing.T) { + // 8 x 1 MiB records exceed the ~1 MiB useful-data cap. A pinned depth must + // win anyway, otherwise a path that needs exactly 8 records is broken by + // an unrelated size heuristic. + c := newTestConn(8, 8) + if got := c.batchCount(1024 * 1024); got != 8 { + t.Fatalf("pinned batch should ignore the 1 MiB cap, got %d", got) + } + // An unpinned batch is still capped. + c = newTestConn(1, 8) + if got := c.batchCount(1024 * 1024); got != 1 { + t.Fatalf("unpinned batch should be capped to 1, got %d", got) + } +} + +func TestAdaptiveBatchStopsAtFloor(t *testing.T) { + c := newTestConn(4, 32) + seen := map[int]bool{} + for i := 0; i < 12; i++ { + _, next, _ := c.shrinkPipeline() + seen[next] = true + } + if c.pipeline != 4 { + t.Fatalf("batch fell to %d, want the floor 4", c.pipeline) + } + if !seen[16] || !seen[8] { + t.Fatalf("expected halving through 16 and 8, saw %v", seen) + } + for i := 0; i < 100; i++ { + c.growPipeline() + } + if c.pipeline != 32 { + t.Fatalf("batch grew to %d, want the ceiling 32", c.pipeline) + } +} + +func TestSingleBatchIsFixed(t *testing.T) { + c := newTestConn(1, 1) + if !c.pinnedBatch() { + t.Fatal("a 1..1 batch must be treated as pinned") + } + c.growPipeline() + if c.pipeline != 1 { + t.Fatalf("batch of 1 grew to %d", c.pipeline) + } +} + func TestReconnectZeroMeansPersistent(t *testing.T) { lane := newRequestLane("127.0.0.1:1", 0, 0, 0) if lane.reconnectEvery != 0 { diff --git a/core/cmd/dragontcp-client/main.go b/core/cmd/dragontcp-client/main.go index 1a15943..d612797 100644 --- a/core/cmd/dragontcp-client/main.go +++ b/core/cmd/dragontcp-client/main.go @@ -374,7 +374,8 @@ func main() { chunkAdaptLog = flag.Bool("chunk-adapt-log", true, "print adaptive chunk size changes") chunkSizeLegacy = flag.Int("chunk-size", 0, "legacy fixed chunk size; nonzero disables adaptation") chunkPollers = flag.Int("chunk-pollers", 1, "reserved compatibility setting; binary transport uses one download worker") - chunkConcurrency = flag.Int("chunk-concurrency", 1, "maximum adaptive download pipeline depth (1-256); 1 keeps concurrency fixed at one") + chunkConcurrency = flag.Int("chunk-concurrency", 1, "maximum download records per request (1-256)") + chunkConcurrencyMin = flag.Int("chunk-concurrency-min", 1, "minimum download records per request (1-256); equal to --chunk-concurrency pins the depth and disables batch adaptation") chunkReconnect = flag.Int("chunk-reconnect-every", 0, "force reconnect after N logical requests; 0 = persistent/automatic") chunkPollDelay = flag.Duration("chunk-poll-delay", 2*time.Millisecond, "delay after an empty chunk poll") chunkTimeout = flag.Duration("chunk-timeout", 2*time.Second, "per-record transaction timeout before adaptive shrink") @@ -417,6 +418,14 @@ func main() { fmt.Fprintln(os.Stderr, "--chunk-concurrency must be between 1 and 256") os.Exit(2) } + if *chunkConcurrencyMin < 1 || *chunkConcurrencyMin > 256 { + fmt.Fprintln(os.Stderr, "--chunk-concurrency-min must be between 1 and 256") + os.Exit(2) + } + if *chunkConcurrencyMin > *chunkConcurrency { + fmt.Fprintln(os.Stderr, "--chunk-concurrency-min must not exceed --chunk-concurrency") + os.Exit(2) + } if *chunkReconnect < 0 { fmt.Fprintln(os.Stderr, "--chunk-reconnect-every must be 0 or greater") os.Exit(2) @@ -433,6 +442,7 @@ func main() { pollDelay: *chunkPollDelay, txnTimeout: *chunkTimeout, tcpBuffer: *tcpBuffer, + minPipeline: *chunkConcurrencyMin, maxPipeline: *chunkConcurrency, } @@ -450,15 +460,21 @@ func main() { fmt.Printf("remote DragonTCP endpoint=%s\n", serverAddr) fmt.Printf("max_connections=%d transport=%s tcp_buffer=%d\n", *maxConnections, *transport, *tcpBuffer) if *transport == "chunk" { + batchMode := "adaptive" + if *chunkConcurrencyMin == *chunkConcurrency { + batchMode = "pinned" + } fmt.Printf( - "adaptive_chunk=%v start=%d min=%d max=%d grow_after=%d pollers=%d concurrency=%d reconnect_every=%d timeout=%s\n", + "adaptive_chunk=%v start=%d min=%d max=%d grow_after=%d pollers=%d batch=%d-%d(%s) reconnect_every=%d timeout=%s\n", *chunkAdaptive, *chunkStart, *chunkMin, *chunkMax, *chunkSuccesses, *chunkPollers, + *chunkConcurrencyMin, *chunkConcurrency, + batchMode, *chunkReconnect, chunkTimeout.String(), )