Privacy and Equity of Federated Personalization for Immersive Cultural Experiences in Autonomous Vehicles

As driving shifts to automated systems, the vehicle cabin is becoming a space for cultural experience. Virtual reality (VR) already lets audiences tour museums and heritage sites remotely [1], so autonomous vehicles could become a new channel through which cultural institutions reach audiences. Streaming such content to a moving vehicle demands high bandwidth and low latency despite frequent handovers [2, 3]. Beyond-5G and 6G networks aim to supply this capacity through integrated sensing and communication [4], unmanned aerial vehicle (UAV)-enabled hybrid free-space optical/radio-frequency (FSO/RF) links [5], and reliable optical access networks [6]. Vehicular edge computing (VEC) addresses this by caching content at roadside units (RSUs) and in vehicles [2], and proactive caching predicts which tiles a passenger will view and where the vehicle will be, placing those tiles at the edge before they are requested [7].

In cultural settings, the data behind this prediction is unusually sensitive. Head-motion traces identify individuals [8], and gaze from eye-tracking headsets can qualify as biometric data [9]. What a passenger watches can reveal potentially sensitive cultural preferences (for example, religious belief, ethnic heritage, or political opinion), and trajectories reveal home, workplace, and routine. These correspond to the special categories of Article 9 of the EU General Data Protection Regulation (GDPR) [10] and to the sensitive personal information of Article 28 of China's Personal Information Protection Law (PIPL), which includes biometrics, religious belief, and whereabouts [11]. Combined in one record, they enable cultural profiling that neither stream allows alone.

Federated learning (FL) reduces this exposure by training on vehicles and sharing only model updates [12]. This paper builds on FedVR360 [13], which originally proposed the predictor studied here and evaluated it against membership inference, black-box attribute inference, and gradient inversion. Because the serving RSU observes individual client updates, record-level differential privacy-which bounds what a single training sequence reveals-does not by itself protect properties shared across a client's records, such as the dominant content category. Using FedVR360 as-is, we ask three questions that matter for cultural services. First, can a stronger adversary, the serving RSU, which observes individual client updates, infer what kind of content a passenger watched? Second, does privacy-preserving personalization distribute accuracy equitably across content categories and passengers? Third, which exposures persist after FL and differential privacy (DP), in particular in the content requests a vehicle must send at delivery time? Our contributions are as follows.

• A cultural sensitivity model that couples biometric, cultural, and spatial exposure, maps it to GDPR Article 9 and PIPL Article 28, and separates training-time from delivery-time exposure.
• A content-category inference attack by an honest-but-curious RSU, evaluated under FedAvg, DP-FedAvg, and FedVR360 with and without personalization, and a discussion of client-level DP and secure aggregation as stronger but costlier safeguards.
• A disaggregated evaluation of accuracy across content categories and passengers, testing whether privacy-preserving personalization serves all passengers comparably.

Immersive content and its data. Virtual and augmented reality are established tools for cultural heritage, supporting virtual museums, reconstructed sites, and remote access to collections [1]. Tile-based 360-degree streaming depends on viewport prediction, now performed with recurrent [14] and transformer [15] models. The motion data these models consume is identifying: Miller et al. re-identified 95% of 511 users from less than five minutes of head and body tracking recorded during 360-degree video [8]. No prior work studies cultural inference for in-vehicle viewing, where head motion and location are recorded together.

Edge caching and FL for vehicular VR. VEC caches content at RSUs and in vehicles to meet VR latency budgets [2], and vehicle-to-everything (V2X) bandwidth allocation can prioritize real-time in-vehicle video [3]. DeepEdge360 [7] jointly predicts viewport and trajectory for tile-level caching but trains centrally on pooled data. FL avoids pooling [12] and has been applied to vehicular trajectory prediction [16] and to Internet-of-Drones security [17], and client-level DP bounds what any single participant's update reveals [18]. Secure aggregation [25] complements client-level DP by preventing the server from observing individual updates at all, though it does not by itself bound what an aggregate reveals. FedVR360 [13] combines prototype-based cross-modal fusion, hierarchical aggregation, differentially private stochastic gradient descent (DP-SGD), and on-device personalization for joint viewport–trajectory prediction. Its evaluation targets membership, black-box attributes, and sample reconstruction, leaving open whether an RSU can infer a passenger's content category from individual updates and how accuracy is distributed across content categories.

Inference attacks and fairness under privacy. Membership inference determines whether a record was used in training [19]. Property inference is more relevant to cultural data: FL updates leak properties of a participant's data unrelated to the learning task, and record-level DP does not prevent inference of properties shared by many of a participant's records [20]. A passenger's preferred content category is such a property; in our experiments we use generic video genres as a statistical proxy for culturally sensitive categories, a limitation we revisit in Section VII. DP-SGD also reduces accuracy disproportionately for underrepresented groups [21], which could degrade personalization for minority audiences; this cost has not been measured for vehicular VR.

A cultural content provider supplies tiled 360-degree content together with metadata such as its content category; RSUs cache these tiles and stream them to vehicles on request. Figure 1 summarizes this architecture (top), the federated learning pipeline (middle), and the threat model (bottom).

We consider the three-tier vehicular edge architecture of [7], [13]. Each vehicle Vᵢ ∈ 𝒱 =V_1,…,V_N carries a head-mounted display that samples head orientation θᵢ(t) (yaw, pitch, roll) and onboard sensors that report position pᵢ(t) ∈ ℝ², velocity, and heading; an onboard unit trains and runs the model locally. RSUs R = {R1, …, RM} cache tiles, serve vehicles within coverage, and aggregate their model updates over intermittent links; a cloud coordinator aggregates RSU-level updates into the global model.

Each vehicle holds a private dataset 𝒟ᵢ = {(Xᵢ⁽ᵏ⁾, Yᵢ⁽ᵏ⁾)}ₖ₌₁ⁿᵢ, where Xᵢ⁽ᵏ⁾ is a window of w past steps of head-orientation and kinematic features and Yᵢ⁽ᵏ⁾ contains the next H steps of viewport tiles (binary vectors over an equirectangular grid) and vehicle positions. A joint model Ŷ = fΘ(X) is trained by FL to minimize ∑ᵢ (nᵢ/n) Fᵢ(Θ), where Fᵢ is the expected multi-task loss on 𝒟ᵢ and n = ∑ᵢ nᵢ. Let 𝒞 be a set of content categories and cᵢ ∈ 𝒞 the dominant category viewed in 𝒟ᵢ. In cultural services, 𝒞 may separate religious, ethnic-heritage, or political content, making cᵢ the attribute most in need of protection. cᵢ is not a model input; it is implicit in the viewing behavior from which the model learns.

A passenger's record is sensitive along three coupled dimensions. Biometric: head motion identifies individuals, and gaze from eye-tracking headsets can qualify as biometric data under the GDPR [9]. Cultural: what a passenger watches, and for how long, can reveal religious belief, ethnic origin, or political opinion, all special categories under GDPR Article 9 [10]. Spatial: trajectories reveal home, workplace, and routine; PIPL Article 28 lists whereabouts alongside biometrics and religious belief [11]. The dimensions compound, linking a record to a person, a place, and beliefs, so protection must cover not only raw records but also attributes, such as cᵢ, that can be inferred from them.

Table 1 lists where this information is exposed, and the bottom panel of Figure 1 summarizes what an adversary observes during training and delivery. Adversaries are honest-but-curious: RSUs, the cloud coordinator, and any party with query access to the released model follow the protocol but try to infer private information. Kmin is the minimum number of contributing clients before an RSU forwards an aggregate. The last row of Table 1 lies outside what FL can address (Section VII). We evaluate two attacks: membership inference [19], which tests whether a training sequence was used, and content-category inference, the focus of this paper, in which the serving RSU recovers ci from the updates it receives (Section IV-B). Random guessing achieves 1/|C|.

Per-example clipping and Gaussian noise in the DP-SGD of [13] give each client record-level rather than user-level (ε, δ)-DP: for datasets Di and Di′ that differ in one training sequence, and any output set S,

This bounds what a single sequence reveals, and hence membership inference, but not a property shared by many of a client's sequences, such as ci [20]; bounding that requires client-level DP [18]. Overlapping windows further weaken the protection of individual time steps. Whether prototype fusion, hierarchical aggregation, and personalization nonetheless suppress category inference is an empirical question that Section VI answers.

FedVR360 [13] encodes trajectory with a one-dimensional convolutional neural network (CNN) followed by a long short-term memory (LSTM) network, and viewport with the same encoder plus multi-head self-attention. The branches interact through K learnable global prototypes per modality: each client attends from its viewport embedding to the viewport prototypes and adds the correspondingly weighted trajectory prototypes, sharing cross-modal context without transmitting embeddings. Shared parameters ΘS (lower encoder layers, fusion module, prototypes) are learned federatively; personal parameters ΘPi (upper encoder layers and biases) are fine-tuned locally and never transmitted.

In each round, a participating vehicle updates ΘS with ΘPi frozen using DP-SGD [22], clipping each per-example gradient to norm C and adding Gaussian noise of standard deviation σC to their sum. The serving RSU aggregates the updates received within a time window, weighted by data volume, and forwards the aggregate only if at least Kmin vehicles contributed; the cloud then aggregates across RSUs. Privacy loss is tracked with a Rényi DP accountant. Because the personal layers are fine-tuned without noise and are used when computing shared gradients, the reported ε covers the shared-parameter updates but is not an end-to-end guarantee, a further reason to measure leakage empirically.

Algorithm: Content-category inference by an honest-but-curious RSU

Require: Global parameters {Θₛ⁽ᵗ⁾}; observed target updates {ΔΘᵢ⁽ᵗ⁾}; auxiliary clients a ∈ 𝒜 with data 𝒟ₐ and categories cₐ; replicates R.

Ensure: Inferred categories {ĉᵢ}.

1: For each round t

2:  For each a ∈ 𝒜 and r = 1, …, R

3:    ΔΘₐ,ᵣ⁽ᵗ⁾ ← LocalTrain(Θₛ⁽ᵗ⁾, subset of 𝒟ₐ) (same (C, σ) as targets)

4:  End For

5:  Fit PCAₜ and classifier gₜ on {(ΔΘₐ,ᵣ⁽ᵗ⁾, cₐ)}

6:  ĉᵢ⁽ᵗ⁾ ← gₜ(PCAₜ(ΔΘᵢ⁽ᵗ⁾)) for each target i observed in round t

7: End For

8: ĉᵢ ← majority vote over {ĉᵢ⁽ᵗ⁾}ₜ for each target i

9: Return {ĉᵢ}

Following the passive property-inference attack of [20], the serving RSU, which relays Θₛ⁽ᵗ⁾ and receives each participant's update ΔΘᵢ⁽ᵗ⁾, also controls auxiliary clients 𝒜 with known categories, such as test vehicles operated by the service provider. Because the training procedure is public, the RSU reproduces local training on auxiliary data with the same DP-SGD configuration (C, σ) as the targets and learns how updates differ by category shown in Algorithm 1. Updates are flattened, reduced by principal component analysis (PCA), and classified by a per-round multinomial logistic-regression model; each target's predictions are combined by majority vote over the rounds in which it participated. Repeating the attack after retraining at each σ yields attack accuracy as a function of ε.

Let F1ᵢ be the F1 score of client i's top-10 predicted viewport tiles (F1@10) and Vc the clients whose dominant category is c. We report category-level accuracy and the gap between the best- and worst-served categories,

together with the mean F1@10 of the worst-served 20% of clients. Comparing these quantities across methods shows whether DP widens disparities [21] and whether personalization narrows them.

Data. Viewport traces come from the dataset of Wu et al. [23], in which 48 participants watched 18 360-degree videos on an HTC Vive headset. Quaternions are converted to yaw, pitch, and roll, angular velocities are derived, traces are downsampled to 10 Hz, and targets are binary activations over an 8 × 8 equirectangular tile grid. Trajectories are generated with SUMO [24] on a 3 km two-way arterial corridor 50 km/h with side roads 30 km/h, a fleet of 80% cars, 15% vans, and 5% buses, and full 60%, partial 20%, and side-road 20% routes; positions are sampled at 10 Hz and normalized to [0, 1], and five RSUs cover the corridor.

Categories and clients. Each participant is paired with one trajectory, giving 48 clients. We use the dataset's video genres, C = {Sports, Performance, Talkshows}, as a proxy for cultural content categories: genre preference is less sensitive than religious or ethnic preference but plays the same statistical role in the attack. Each client is assigned a dominant category ci, balanced across C, and 80% of its sequences come from videos of ci. Twelve clients (four per category) act as the adversary's auxiliary clients; all 48 clients train, and the other 36 are the attack targets. Sequences are split chronologically into 80% training and 20% test data, with w = 20 input steps and a horizon of H = 30 steps (3 s). No public dataset captures in-vehicle viewing and mobility from the same subjects, so the results are a controlled proof of concept.

Following [13], both branches produce 64-dimensional embeddings, the viewport branch uses four attention heads, and there are K = 16 prototypes. Training runs for 20 rounds with 10 uniformly sampled clients per round, one local epoch, batch size B = 32, learning rates of 0.001 (shared) and 0.01 (personal), and loss weights of 1.5 (viewport) and 1.0 (trajectory). All sampled clients report within the round window, so staleness discounting is inactive, and Kmin = 2. DP-SGD uses C = 1.0 and δ = 10⁻⁵. We sweep σ ∈ {0.3, 0.5, 0.8, 1.2} and report the largest per-client ε from a Rényi accountant for the subsampled Gaussian mechanism with sampling rate q = B/ni. Main results use σ* = 0.5 (ε* ≈ 3.0); Figure 1 reports the sweep. All results are averaged over three seeds and reported as mean ± standard deviation.

Centralized: the same architecture trained on pooled data, a utility reference that exposes raw records by construction and is therefore not attacked. FedAvg [12]: federated training without DP or personalization. DP-FedAvg: FedAvg with the same record-level DP-SGD as FedVR360. FedVR360 [13], with and without personalization (all parameters shared). All federated methods use identical client sampling, data splits, and σ. Utility is measured by viewport F1@10, trajectory mean absolute error (MAE), and the measures of Eq. (2). Privacy is measured by category-attack accuracy over the targets, compared with chance using a one-sided binomial test, and by shadow-model membership inference [19], where 50% is chance. The attack uses R = 20 replicates per auxiliary client per round, 50 PCA components, and ℓ₂-regularized logistic regression.

Unless stated otherwise, σ = 0.5 (ε = 3.0). Differences smaller than the seed-to-seed standard deviation are not interpreted.

Centralized training sets the reference (F1@10 = 0.840, MAE = 0.274; Table 1). FedVR360 reaches F1@10 = 0.795, 4.5 percentage points below this reference, compared with 8.2 without personalization, 10.5 for FedAvg, and 12.8 for DP-FedAvg. Record-level DP widens the category gap ΔC from 0.052 (FedAvg) to 0.069 (DP-FedAvg), consistent with [21], whereas on-device personalization narrows it from 0.058 to 0.046 and raises worst-20% F1 from 0.712 to 0.741.

Record-level DP behaves as its definition predicts: membership inference falls from 62.04% (FedAvg) to 50.00% (FedVR360), at chance, yet the serving RSU still recovers each passenger's dominant content category with 50.93% accuracy against 33.3% chance (p ≤ 0.017 in every seed; Table 2). Protecting individual sequences therefore does not protect content preference, the attribute that exposes cultural identity [20]. Figure 2 shows that category-attack accuracy falls to 47.22% at ε = 2.2, the lowest value swept and still above chance (p = 0.038), while FedVR360's F1@10 drops to 0.745 ± 0.011, 5.0 percentage points below its value at ε = 3.0. Suppressing category leakage through noise alone is therefore costly.

Personalization without central profiles. With FL, raw records stay in the vehicle and only shared-parameter updates are transmitted; personal layers are fine-tuned locally and never leave the device. A cultural institution therefore receives only an aggregate model. This protection does not extend to content preference, which remains inferable from the updates a vehicle must send, even at our strongest evaluated setting (ε = 3.0). Institutions can personalize without holding raw data but should treat the preference signal as exposed, for example by not logging or acting on inferred categories. This matters because cultural content has traditionally been consumed in spaces, such as museums, where visitors choose what to view without being individually recorded.

Delivery-time exposure. FL does not hide what is streamed: tile requests and proactive prefetching reveal content, predicted viewport, and location to the RSU and the content server (Table 1). Caching complete popular titles at RSUs, mixing requests across vehicles, and private information retrieval all reduce this exposure, at a bandwidth or latency cost that real-time VR makes significant. Full-title caching is the most practical near-term option, whereas private information retrieval offers the strongest guarantee but is too costly for real-time 4K VR. We do not evaluate these mitigations here; our contribution is to identify delivery-time exposure as a first-class problem distinct from training-time leakage. A quantitative comparison of full-title caching, request mixing, and private information retrieval under realistic VR latency budgets is left as dedicated future work. We regard delivery-time request privacy as the main open problem for private cultural streaming.

Limitations. Our categories are generic video genres viewed by 48 participants, a proxy that cannot represent the range of cultural content or audiences; culturally diverse datasets, built with cultural institutions, are needed. Viewport and trajectory data come from different subjects and are paired synthetically, the adversaries are honest-but-curious rather than active, and the evaluation covers 48 clients and five RSUs. No public dataset captures in-vehicle viewing and mobility from the same subjects, so our absolute numbers should be read as a controlled proof of concept rather than field estimates. Client-level DP [18] would bound category inference formally, but against an RSU that observes individual updates the noise must be added per vehicle, at an accuracy cost that we leave to future work. Secure aggregation [25] would hide individual updates from the RSU entirely but does not by itself bound what the aggregate reveals.

This paper examines whether federated learning protects the cultural attributes exposed when immersive experiences are personalized in autonomous vehicles, using generic video genres as a statistical proxy for culturally sensitive categories. Record-level DP suppressed membership inference but not content-category inference: a curious RSU recovered passengers' dominant category with 50.93% accuracy against 33.3% chance at ε = 3.0, while FedVR360 stayed within 4.5 percentage points of centralized F1@10. DP widened the accuracy gap across content categories, and on-device personalization narrowed it. For cultural services, privacy-preserving personalization is therefore a precondition for deployment, and it must cover both what passengers' data reveals during training and what their requests reveal during delivery. Because the serving RSU observes individual updates, record-level DP cannot bound category inference; client-level DP and secure aggregation are the natural next step. Future work will also address delivery-time request privacy and culturally diverse in-vehicle datasets.