If you're talking about using machine learning to create a model (rather than hard coding one) in order to trade algorithmically, then yes I have done. My experience was it's very difficult
Basically you are introducing a lot more complexity than regular algo trading because now you have to create, regularly retrain/update and deploy the model. My day job was an an ML architect, and I previously worked in the hedge fund industry so I consider myself fairly well qualified to do this. I'm piggy backing it on concepts and data that I have been using for some time and its is still difficult. Still experimenting with it but it is challenging.