--- layout: post title: "Using webcam to control Live2D Cubism model in Linux" date: 2021-01-11 05:57:03 +0800 --- TL;DR: Code: [GitHub][github-face], [self-hosted][self-face], demo: video below I was researching how to get FaceRig to work on Linux, and eventually concluded that it is pretty much impossible. So I thought, hey, why not just code it myself? It can't be that difficult, right? I mean, it probably won't be as feature-rich as the original, but all I need / want is the basic "avatar moving along with my face" functionality. So I spent a couple weekends, digging up the (very poorly translated) Live2D docs (but still, kudos to Live2D Inc for actually translating them), struggled with some questionable anti-patterns, and arrived with this: [(Signature)][sig0] Also available on [YouTube][ytdemo] Source code available on [GitHub][github-face] and [my own server][self-face]. Yes the code is kind of messy and could do some refactorings, but it works so I couldn't be bothered. ¯\\\_(ツ)_/¯ With some input and suggestions from another user I have also created a spin-off project, where instead of using facial tracking, the avatar is controlled by mouse cursor position and customizable via a UI. Some more demos (not really that important so I haven't bothered self-hosting them): - [Comparing facial tracking and mouse tracking][comparedemo] - [CLI controls for mouse tracking][clidemo] - [GUI controls for mouse tracking][guidemo] Source code for this spin-off project: [GitHub][github-mouse], [self-hosted][self-mouse]. Please feel free to get in touch for any comments, suggestions, bug reports, questions, or any kind of feedback! [sig0]: /blog/sig/2021-01-11-linux-live2d.0.mp4.sig [ytdemo]: https://www.youtube.com/watch?v=SZPEKwEqbdI [comparedemo]: https://www.youtube.com/watch?v=MA7H3v9dtxM [clidemo]: https://www.youtube.com/watch?v=xFoLu5Dm-tg [guidemo]: https://www.youtube.com/watch?v=7N1yRgMnQMw [github-face]: https://github.com/adrianiainlam/facial-landmarks-for-cubism [self-face]: https://adrianiainlam.tk/git/?p=facial-landmarks-for-cubism.git;a=summary [github-mouse]: https://github.com/adrianiainlam/mouse-tracker-for-cubism [self-mouse]: https://adrianiainlam.tk/git/?p=mouse-tracker-for-cubism.git;a=summary