Shintaro Shiba

Research Scientist, Ph.D in Event-based Computer Vision

I’m Research Scientist in computer vision (especially event-based camera). I’m now trying to create a momentum where event cameras get into the real-world applications and products, by bridging academic research and industrial product works.

I got my Ph.D. in Engineering, focusing on event-based motion estimation, with the supervision by Prof. Guillermo Gallego (TU Berlin, Germany) and Prof. Yoshimitsu Aoki (Keio University, Japan). Event cameras are the very intersection of my different backgrounds: neuroscience, machine learning, and autonomous driving. My original background is in neuroscience as Bachelor’s and Master’s degrees. I wanted to study animal brains before rushing into the AI field, since they are very efficient, robust, and still accurate in recognizing the world. My pre-doctoral professional experience is machine-learning software engineer in production-scale MLOps (ML and data pipeline) for autonomous driving at Woven by Toyota. There I learned practices to convert ML researches into products to solve the real-world problems. Please see my CV for more details, as I’m happy to talk about possible opportunities in a team trying to pioneer event-camera research and/or development.

Je parle un peu le français and spreche ich etwas Deustch. Apart from work, I love to play jazz saxophone and dance :)

イベントカメラ(イベントベースカメラ・イベントビジョンセンサー・ダイナミックビジョンセンサー)に関するコンピュータビジョン研究と開発をしています。 元々、神経科学を修士まで勉強し、生物のように頑健で電力効率の高い視覚認知システムの実現についての興味がありました。 同時に、自動運転用の認識ソフトウェア・機械学習開発をおこなった経験の中で、 従来のフレームベースカメラに存在するモーションブラーやダイナミックレンジ、データや消費電力効率の悪さに、産業面からの長期的な課題意識を持ちました。 そこでイベントカメラに着目し、研究に取り組み、分野のトップ研究室であるベルリン工科大に留学し、博士(工学)を取得しました。 研究では、日本国内で数々の賞をいただきましたが、 この技術を産業と結びつけ、マーケットを創出するために、 産・学を飛び越えて挑戦をしています。

news

Sep 28, 2024 Looking forward to attending ECCV 2024, Milan! I have two poster presentations at Workshop on Neuromorphic Vision Workshop on Sunday.
Aug 1, 2024 On 5th August I will give an invited talk and poster presentation at 27th Meeting on Image Recognition and Understanding (MIRU2024), Japan, about the PAMI paper “Event-based Background-Oriented Schlieren”.
Jun 1, 2024 On 14th June, I will give an invited talk and poster presentation at Symposium on Sensing via Image Information (SSII) 2024, Japan.
May 4, 2024 Our new paper is accepted to IEEE T-PAMI! “Secrets of Event-based Optical Flow, Depth and Ego-motion Estimation by Contrast Maximization”, Paper. I will update the reposiroty.
Apr 15, 2024 Our new paper “3D Human Scan With A Moving Event Camera” is accepted to IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshop On Computer Vision For Mixed Reality (CV4MR), Seattle, 2024! Paper.
Mar 25, 2024 I am honered to receieve the three awards related to my Ph.D. research, such as: Dean’s Award at Keio, The Telecommunication Advancement Foundation, and Japanese Society of Automotive Engineering. In total, I received four awards including the Ikushi prize. I deeply appreciate my collaborators, friends, and family.
Feb 25, 2024 On March 5th, I will give an invited talk at Next Generation Image Acquisition and Vision System (次世代画像入力ビジョンシステム部会), which is a part of JTTAS (日本工業技術振興協会).
Feb 3, 2024 Mr. Kai Kohyama, a Bachelor student that I have been supervising, successfully finished his Bachelor thesis (Keio Uni, Faculty of Science and Technology)! The title and details are to be updated, as we prepare a submission.
Jan 20, 2024 I am honered and excited to receieve the 14th JSPS Ikushi Award!! (details) JSPS (Japan Society for the Promotion of Science) established the JSPS Ikushi Prize program in 2010. It functions to formally recognize outstanding doctoral students who can be expected to contribute to Japan’s future scientific advancement. About 16 recipients are chosen for the prize each fiscal year, receiving a certificate, a medal, and a purse of 1.1 million yen.
Dec 1, 2023 The paper “Event-based Background-Oriented Schlieren” is now featured at the top page of the TUBerlin webpage. Yay! News
Oct 24, 2023 Our new paper is accepted to IEEE T-PAMI! “Event-based Background-Oriented Schlieren”, Paper and Code.
Aug 30, 2023 Mr. Yannick Klose, a Master student that I have been supervising, successfully finished his Master thesis (TUBerlin Faculty IV)! The title is to be updated (A part of his work is under review).
Jul 11, 2023 I will give an invited talk at ACRO, Teikyo University, on July 12th. Details to be updated.
Jun 14, 2023 See you all at CVPR 2023, Vancouver! I have a poster presentation at Event Vision Workshop (19th Mon AM/PM). Looking forward to seeing people in the event-based vision community!
Jun 7, 2023 I will give an invited talk at Image Processing Group, Universitat Politècnica de Catalunya, on June 12th.
May 18, 2023 I finished the defense successfully! I appreciate the thesis committee, including Prof. Aoki at Keio, Prof. Gallego at TUBerlin, Prof. Daniilidis at UPenn, and all audience for the constructive and useful discussions.
May 2, 2023 I will do my defense on May 17. It’s online, so please contact me if you are interested in joining. Also, I will give an invited talk at SCIoI, TU Berlin, on May 25th.
Mar 10, 2023 Mr. Ryo Yamaki, an undergrad student that I have been supervising, successfully finished his Bachelor’s thesis! The title of the thesis is “Motion Segmentation for a car-mounted event camera”.
Feb 28, 2023 Our paper is featured at Keio Research News Lettter, “The Penmark” on February! Keio Researchers Co-Develop Bio-Inspired Robotic Eyes (Original paper)
Jan 29, 2023 Our paper is featured at Advanced Science News for outreach! Bio-inspired robotic eyes that better estimate motion (Original paper)
Jan 10, 2023 Our new paper is out from IEEE Signal Processing Letters! “Fast Event-Based Optical Flow Estimation by Triplet Matching”, link.
Dec 19, 2022 Our new paper is out from Advanced Intelligent Systems! “A Fast Geometric Regularizer to Mitigate Event Collapse in the Contrast Maximization Framework”, arxiv.
Dec 9, 2022 I got the best paper award (小田原賞) at ViEW 2022 (Vision Engineering Workshop 2022, Japan)!
Nov 14, 2022 I give an invited lecture on November 17 (Thu) at Active-Percption course, Faculty IV, TU Berlin.
Nov 7, 2022 Our work about motion and optical flow estimation from events is accpected as an oral presentation at ViEW 2022 (in Japanese). Paper details coming soon.