Nowadays, the training phase of AI -- which is highly compute-intensive -- is being increasingly sped up by cloud computing. However, communicating with the cloud introduces latency during AI inference phase, which leads to poor performance for applications requiring 'hard' real-time responses from an on-premise location. We define "hard real-time" as a microsecond timescale, end-to-end, for a single batch of data communication and data processing. We propose Edge Neural Network (ENN), a distributed architecture for accelerated DNNs on the edge. The main objective of the ENN project is to design and build an AI software and hardware system with real-time inference capabilities for edge applications that exhibit high-speed, massive dataset characteristics wherein communicating with the cloud directly would be impractical or too expensive.