ETD

• CIFAR-10
• Differentiable Architecture Search
• DoReFa
• GTSDB
• Learned Step Size Quantization
• NNI Toolkit
• Photonic Neural Network
• Quantization-Aware Training

This thesis addresses a gap in the design of neural networks intended for deployment on photonic computing hardware. Photonic Neural Networks (PNNs) offer high-bandwidth, low-latency, energy-efficient inference by performing computation optically, but they are constrained by hardware non-idealities: a limited Effective Number of Bits (ENOB, typically ≤6), strictly non-negative inputs, restricted neuron fan-in (≤200), and small convolutional kernel sizes (≤3×3). Existing Photonic-Aware Neural Network (PANN) approaches address these constraints through manual architecture design, applying quantization only as a post-processing step after the architecture has already been fixed under full-precision assumptions, leaving architecture-level optimization disconnected from the physical realities of the hardware.
To close this gap, the thesis introduces QA-DARTS, a Quantization-Aware Differentiable Architecture Search framework that integrates quantization directly into the architecture search process rather than applying it afterward. Built on top of Microsoft's NNI toolkit, the framework uses custom mutation hooks to replace NNI's standard mutable convolutional and linear layers with quantization-aware mixed operations, allowing DARTS to explore candidate architectures while weight and activation quantization are already active. Two quantization algorithms are implemented and compared: DoReFa, which applies fixed quantization ranges via tanh-based weight normalization, and Learned Step Size Quantization (LSQ), which learns adaptive per-channel quantization step sizes jointly with network weights. Both rely on a Straight-Through Estimator to allow gradients to flow through the non-differentiable rounding operation. The search space is built around seven residual blocks, each offering a choice between standard convolution, depthwise-separable convolution, or a residual-only operation (the latter restricted to alternating layers to avoid architectural collapse), along with two downsampling strategies at three points in the network.
The framework is evaluated on CIFAR-10 and GTSDB (adapted to a classification task) at 6-bit, 4-bit, and 2-bit precision. A direct comparison between DoReFa and LSQ at 6-bit on CIFAR-10 shows LSQ achieving both stronger architecture parameter polarization and substantially higher final accuracy (90.44% vs. 82.66%), leading to LSQ being adopted exclusively for all subsequent experiments. Across precision levels, the discovered architectures show graceful degradation: on CIFAR-10, accuracy drops from 90.44% (6-bit) to 88.22% (4-bit) to 75.55% (2-bit); on GTSDB, from 97.23% (6-bit) to 94.74% (4-bit) to 87.81% (2-bit). The 2-bit regime required specific stabilization measures — network widening and a full-precision warmup phase before activating quantization — since direct training from a quantized state failed to converge, as demonstrated by an ablation comparing warmup and no-warmup runs.
A dedicated analysis chapter examines the systematic gap between the supernet's validation accuracy during search and the retrained discrete architecture's test accuracy, attributing it to three mechanisms: a "mixed operation advantage" (the supernet behaves like an ensemble of weighted operations during search), weight co-adaptation (search-time weights are specialized for a relaxed architecture, not a fixed one), and overfitting. The relative contribution of each factor is shown to shift with bit-width and dataset size: on CIFAR-10, the mixed-operation advantage grows in importance as precision decreases while overfitting recedes (since lower-precision networks have less capacity to memorize training data), whereas on the smaller GTSDB dataset, overfitting remains the dominant factor even at 2-bit precision.
The thesis concludes that integrating quantization into the architecture search process, rather than treating it as a post-hoc step, produces architectures that are inherently more robust to aggressive precision reduction, supporting the central hypothesis motivating the work. Limitations include the restriction to two relatively small image classification benchmarks, purely simulated hardware constraints (no physical photonic validation), and a search space that does not explore mixed-precision strategies or explicit energy/latency objectives. Future work directions include extending the framework to other task domains such as time-series/signal-processing workloads relevant to telecommunications, validating discovered architectures on physical photonic hardware, and exploring reconfigurable, optically-trained architectures enabled by Optical Circuit Switching.