← All AI/ML open problems

Privacy & encrypted learning · Learning dynamics & scaling / Open question

Training Networks Entirely on Encrypted Data

Approximation errors feed back into every later gradient, so an encrypted optimizer is a perturbed nonlinear dynamical system rather than just many independent encrypted forward passes. Efficiency payoff: Control accumulated training error with fewer expensive refreshes and lower ciphertext precision.

OpenEditorially scopedFramework needs definitionCompiled 2026-11-20

Seed catalog · Showing the bundled record.

Problem statement

Train fθ(x)=∑j=1majtanh⁡(wj⊤x)f_\theta(x)=\sum_{j=1}^m a_j\tanh(w_j^\top x) by TT full-batch squared-loss gradient steps on nn encrypted examples with ∥xi∥≤1\|x_i\|\leq1, ∣yi∣≤1|y_i|\leq1 and public initialization. The server keeps data, intermediate gradients and updated parameters encrypted. Determine approximation degrees, precision and refresh schedules giving E∥fθ^T−fθT∥L2(DX)2≤ε\mathbb E\|f_{\hat\theta_T}-f_{\theta_T}\|_{L^2(\mathcal D_X)}^2\leq\varepsilon after decryption, where θT\theta_T is the matched plaintext training trajectory, with explicit ciphertext work and memory.

Assumptions and scope

  • Use a client-held secret key, an honest-but-curious server, a public step schedule and only final model decryption. The client supplies encrypted data and evaluation keys; any later client assistance must be reported as an interactive variant.
  • Specify polynomial tanh and derivative approximations, CKKS arithmetic/refresh errors and a bounded forward-invariant parameter/activation region. If enforcing that region needs clipping or projection, include its encrypted circuit and approximation cost; do not assume intermediate plaintext diagnostics.
  • The evaluation law D_X is supported on the unit ball. Match initialization, data, steps and optimizer to the plaintext comparator. State whether error is in expectation or high probability and how range/failure events are handled. Tracking plaintext training does not prove plaintext generalization; add any such theorem separately.

What would settle it?

A concrete encrypted-training protocol with a nonvacuous trajectory/prediction error and cryptographic cost bound, identifying stability conditions for polynomial rather than exponential precision growth in T. Include a counterexample or lower-bound regime for unstable updates.

Selected for a theoretical or computational target. A fitted scaling curve or improved benchmark score alone does not establish the stated learning-theoretic guarantee.

Status and evidence

Non-interactive encrypted neural training already has CPU/GPU implementations. CKKS supplies approximate arithmetic, and HE-aware approximation/weight co-adaptation exists for inference. The new target is a rigorous finite-training perturbation/cost analysis. Encryption hides computation from the evaluator; releasing the decrypted model is not automatically DP. This is an editorial, source-motivated research target. The preliminary review through 2026-11-20 does not certify novelty or exclude an unlocated resolution.

Preliminary source check; not an expert-verified open status or an independent audit of proposed proofs. The compilation date is not a literature-review cutoff.

Sources and further reading

  1. Mihara — GPU-Accelerated Non-Interactive Encrypted Neural Network Training with Batched Diagonal Packing ↗related algorithm · source checked 2026-11-20

    Primary abstract checked: GPU-resident CKKS training/bootstrapping and plaintext-twin experiments. The reported runs are single-initialization measurements; CPU/GPU equality and plaintext generalization comparisons have different scopes.

  2. Cheon, Kim, Kim and Song — Homomorphic Encryption for Arithmetic of Approximate Numbers ↗partial theory · source checked 2026-11-20

    Primary IACR abstract checked: approximate SIMD arithmetic, rescaling and precision management under an RLWE-based construction. Numerical approximation and cryptographic confidentiality are distinct guarantees.

  3. Zirilli et al. — HEAT: Faster Fully Homomorphic Inference via Approximations-Weights Co-Adaptation ↗partial theory · source checked 2026-11-20

    Primary search abstract checked: trainable nonlinearity iteration counts, fixed-weight comparison to quantization-aware training and encrypted GPT-2 experiments. Approximation/weight co-design already exists; uniform full-trajectory guarantees are separate.