Compared with the cameras found in smartphones, computer webcams were all but frozen in time a decade ago, such that most still sport 720p or 1080p resolutions with middling color accuracy and low-light performance. After revealing the hidden low-light potential in older iPhone cameras, NeuralCam is back with a new app that turns Apple’s smartphones into “smart webcams.” And better yet, it’s free — albeit with caveats.
The premise of NeuralCam Live is to combine an iPhone’s front camera with machine learning to create a higher-quality computer video stream than a traditional webcam can deliver. After installing the iOS app and a Mac driver — Windows support is coming soon — the iPhone feeds a live stream to your computer including real-time machine learning- and computer vision-enhanced video. All the video processing is handled on the device rather than on the computer, and the company is developing an iOS SDK so third-party video calling and streaming apps can control the enhancements.
Most users will be interested in the app’s “style” and “enhance” features, as they let computers offer live video with facial smoothing, various color filters akin to Instagram’s, and low-light enhancements that brighten up videos shot in dark environments. The app also has an on-screen meter that evaluates the current lighting conditions to let you know whether your chosen background and lighting will deliver “poor,” “good,” or “great” results to viewers, so you can make adjustments prior to starting a video chat.
The app’s third ML framework is “recognition,” which among other things lets NeuralCam Live instantly blur the camera output if it detects facial touches or nudity. While we didn’t test the latter feature, the former one worked only occasionally and with some lag on an iPhone 11 Pro, suggesting that the machine learning model may still need some optimization work. Recognition also enables “head bubble” and “circle light” modes that automatically locate your face and either black out or white out everything save for a circle around your head — the former to eliminate distracting backgrounds or people, the latter to illuminate your face in a dark room.
While NeuralCam Live is free to download, arriving with almost all of its core functionality unlocked, the company is offering a $5 monthly or $30 annual subscription to access some of its “growing number” of filters and its low-light optimization mode. Another potential caveat is the app’s currently limited support for Mac video conferencing apps: Zoom, Google Meet, and Microsoft Teams are supported, but Apple’s native FaceTime isn’t, nor is the Mac’s Safari browser. The company says you can use Chrome, Firefox, and any conferencing app that supports “virtual webcams.”
Limitations aside, NeuralCam Live is an intriguing app, and the company suggests it’s representative of a broader industry evolution from traditional cameras to deep learning- and computer vision-powered smart cameras. Following its prior NightCam (aka NeuralCam Night Mode) and WebCam (NeuralCam Live) releases, the company is planning PhotoCam, DiagnoseCam, NotesCam, MonitorCam, and TrainingCam apps with various types of smart functionality, eventually going beyond consumer photography and videography to serve industries such as education, health care, and entertainment. It’s increasingly clear that NeuralCam is a company worth watching in the smart camera space, and that we’re only at the beginning of an era where computational photography and videography become ubiquitous rather than passive or alternative options for most people.
Author: Jeremy Horwitz.
Source: Venturebeat
