The slides of my talk can be downloaded from this link
Artificial Intelligence(AI) and Machine Learning(ML) continues to be a hot topic in OFC 2018. In fact, the number of workshops/panels/symposiums/sessions on AI/ML has steadily increased over the past few years.
I was asked to share my views on the role of ML in our community of optical communications and networks in one of the workshops and I had the honor of joining my Ph.D. advisor Prof. Joseph Kahn from Stanford in the panel of speakers. Other expert panelists include researchers and industry leaders from Facebook, Nokia Bell Labs, ADVA, Fujitsu, Infinera and ONF.
I think It should be emphasized again that for linear systems, there's no place for ML as we already have the best possible detection algorithms. ML can potentially play a role in the following areas:
Fiber nonlinearity compensation
Optical Performance Monitoring (OPM)
Soft-fault identification/localization and preventive network maintenance
Cognitive software defined network(SDN) with dynamic traffic routing, modulation and wavelength assignment(RMWA) based on real-time link quality monitoring
I think that #3 will have the biggest positive impact to network operators because it can successfully prevent network disruptions and dramatically reduce the time and labour cost for repair. And this viewpoint seems to be a consensus among most of the participants.
Machine Learning techniques will be most useful in soft-fault identification/localization and preventive network maintenance
Moreover, there's some trends about the AI/ML work in our community that we need to take a cautious note. Firstly, some of the proposed technique are actually standard statistical signal processing (things such as HMM, Bayesian analysis, PCA, regression, Kalman filtering). This is because some of these ML techniques are rooted in the fundamental foundations of Statistics and Probability. If you have a communication theory background, you know a lot of these algorithms already.
Also, as we are seldom 100% clueless about the underlying physics of our problem, we almost certainly can hand-pick features and have semi-analytical models that maps the inputs and outputs. Often times, what's left to be done by the machine learning algorithm is just optimizing a handful of parameters. This is extensive and advanced calibrations at most and calling these calculations as machine learning will be doing a dis-service to real ML researchers.
Consequently, I think its hard to find a problem (or more specifically, a residual problem for AI after we throw in all the underlying physics we know about the situation) that really needs deep learning and all these fantastic and complex neural network structure discovered over the past few years. In a way, the 2 fundamental problems that we are trying to solve, namely 1) maximize transmission capacity from point A to B, and 2) maximize information flow in dynamic networks are "pattern recognition-wise" too simple for those very powerful deep learning algorithms. Rather, a lot of the bottlenecks in our research community stem from insufficient understanding of the underlying physics, materials and hardware device constraints and/or inefficient fabrication process.
Ultimately, the 2 fundamental problems that we are trying to solve in optical communications and networks are "pattern recognition-wise" too simple for those powerful deep learning algorithms.
Conclusions: Traditional and simpler machine learning algorithms like artificial neural network(ANN), support vector machine(SVM), decision trees etc are handy tools that can enable continuous network equipment monitoring and achieve preventive maintenance. Deep learning probably won't be too relevant. We will still see lots of discussions about AI-assisted optical networks, but the focus will be how to obtain relevant data sets and incorporate the insights/recommendation from the ML algorithms into the overall SDN framework. Those are by all means important problems, but not ML problems.