They know a startup's journey, put themselves in the founder's shoes, and are honest with what can and can't be done.
From Vision to Impact
Built to Launch. Engineered to Grow.What Was Done
We partnered with Kanvo to turn an ambitious accessibility concept into a working iPadOS product designed to help people with communication disabilities translate gestures into text.
Our team designed and engineered the product experience around machine-learning-powered gesture recognition, including camera-based gesture capture, recognition results, text-to-speech, user accounts, settings, and supporting communication flows. The core recognition module was successfully integrated into the application.
The engagement went beyond application development. We conducted technical R&D into sign-language recognition, developed an accessibility-focused UI/UX direction, established backend and AWS infrastructure, and designed and developed Kanvo’s landing page to support product validation and fundraising.
Delivered Result
Kanvo progressed from an early concept to a functional Alpha product with machine-learning gesture recognition running inside the iPadOS application and a complete core user flow ready for demonstration to investors and early adopters.
The recognition model was optimized during development, including additional model training to improve how the camera distinguishes hands across different backgrounds, while the product experience was refined around the needs of its accessibility-focused audience.
The foundation was designed for further evolution from character-level recognition toward word and gesture recognition and, ultimately, personalized on-device model training as the product progressed through its planned Beta and MVP stages.
Dataset images in the ASL alphabet dataset used as the foundation for Alpha ML development.
Core digital experiences delivered across the native iPadOS product and web presence.
Recognition evolution stages defined across Alpha, Beta, and MVP.
Authentication methods designed for account access.
Accessibility standard considered during UX/UI research and design.
Clutch rating reflecting client satisfaction with the collaboration.
Project details
Category
- Accessibility
- AI / Machine Learning
- Australia
Clutch Review
“They know a startup’s journey, put themselves in the founder’s shoes, and are honest with what can and can’t be done.”
Project Timeline
Technology Used
- iPadOS: Native
- Swift / SwiftUI
- Apple Vision / AVFoundation
- Core ML
- Create ML
- Backend API
- AWS
- Firebase
- New Relic
9 Team Members
- Business Analyst / PM
- Project Manager
- UI/UX & Design
- iOS Development
- Backend Engineering
- Web Development
- Technical Leadership
About the Project
Kanvo is an accessibility-focused communication platform designed to help people with communication disabilities translate gestures into text using machine learning. The product was conceived around a native iPadOS experience, combining camera-based gesture capture, ML recognition, text-to-speech, and personalized communication flows within one accessible interface.

The Product
Kanvo is an ML-powered communication application that captures users’ gestures through the device camera and translates recognized signs into text, helping reduce communication barriers in everyday interactions.
Project Goal
To validate whether sign-language recognition could become a viable digital product and transform the initial concept into a functional Alpha application ready to demonstrate to early adopters and potential investors.
Responsibilities
Our team covered product discovery, technical R&D, UX/UI design, and development, including ML recognition research and integration, native iPadOS development, backend infrastructure, accessibility-focused design, authentication, and the supporting web landing experience.
Role in Project
We became Kanvo’s product development and technical partner, helping move the idea from early concepts and technical uncertainty to a working Alpha with gesture recognition integrated directly into the iPadOS application.
Workflow Plan
From Technical Validation to Working Alpha
Product & UX
- Product Discovery
- Requirements Analysis
- Accessibility Research
- User Flows
- UX/UI Design
- Inclusive Design
- Alpha Scope Definition
ML & Engineering
- Sign Language Recognition R&D
- ML Dataset Research
- Vision & AVFoundation
- Core ML & Create ML
- Native iPadOS Development
- Backend Development
- AWS Infrastructure
Validation & Delivery
- ML Model Training
- Gesture Recognition Integration
- Recognition Optimization
- TestFlight Builds
- Product Iteration
- Landing Page Development
- Alpha Delivery
Turning Sign Language Recognition Into a Working Product
Kanvo was engineered as an accessibility-focused communication product that uses machine learning to recognize gestures captured through an iPad camera and translate them into text.
The challenge went beyond building a conventional mobile application. The core product depended on proving that sign language recognition could work within a practical user experience, requiring technical research, ML experimentation, camera processing, model training, and accessible UX design before the concept could become a functional Alpha product.
Engineering Challenge
Kanvo’s core functionality could not rely on an existing plug-and-play translation service. Our technical research found no open-source solution, paid API, or ready-made SDK capable of fully covering the product requirements.
- Real-time camera-based gesture capture
- Sign language recognition
- ML model selection & training
- Recognition across different backgrounds
- Gesture-to-text communication
- Accessible interaction design
- Native iPadOS experience
- Foundation for future recognition expansion
The key challenge was to transform an experimental ML capability into a communication flow simple enough to become part of an accessible everyday product.
Architecture & Engineering Solution
We developed a native Apple-based recognition approach combining camera processing with machine learning, while structuring the product so recognition capabilities could evolve through successive development stages.
- Native iPadOS application
- Vision / AVFoundation camera processing
- Core ML recognition
- Create ML model training
- Image-based ASL dataset
- Backend API & AWS infrastructure
- Gesture recognition module
- Text-to-speech flow
- Authentication & user accounts
- Recognition model optimization
This foundation resulted in gesture recognition running inside the Kanvo application and established a technical path toward word-level recognition and personalized on-device model training.
From Sign Language to Digital Communication
Building Kanvo meant turning a complex form of human communication into an accessible digital experience. The challenge was not only to recognize gestures with machine learning, but to make the entire interaction simple enough for people to use in real communication scenarios.
Technical Feasibility
Validate whether gesture recognition could support a practical communication experience without relying on a ready-made translation solution.
Accessible Interaction
Make gesture capture, recognition results, navigation, and communication flows clear and approachable for users with different accessibility needs.
From Recognition to Communication
Turn ML output into something immediately useful through readable text, editing, confirmation, and text-to-speech.
Recognition Recovery
Give users a clear way forward when recognition fails through retry, correction, and personal Library flows.
Turning ML Research Into Product
The Alpha was structured around proving the core recognition concept first, then integrating it into an accessible product experience that could become the foundation for more advanced recognition capabilities.
STEP I: Validate the Recognition Approach
We researched available recognition technologies, selected Apple’s Vision, AVFoundation, Core ML, and Create ML stack, and established an image-based approach for the first version of sign-language recognition.
STEP II: Build the Communication Experience
We designed and developed the native iPadOS flow around gesture capture and recognition, connecting ML output with editable text, text-to-speech, accounts, settings, and supporting communication interactions.
STEP III: Integrate & Optimize the Alpha
The recognition module was integrated directly into the application and iteratively improved through additional model training, including better hand distinction across different backgrounds. This resulted in a working Alpha suitable for product demonstration and further validation.
Understanding the Core Users
We designed Kanvo around communication between people who use sign language and people who may not understand it, with the product acting as a bridge between gestures, text, and spoken communication.

Elena“I want to communicate naturally using gestures and have my message understood without depending on another person to translate.”
The Sign Language User
Uses sign language as a primary or preferred form of communication. They need a simple way to capture gestures, review the recognized message, correct it when necessary, and communicate the result to another person.

Marcus“I want to understand what the person is communicating without needing to know sign language myself.”
The Communication Partner
Does not necessarily understand sign language but needs to communicate with someone who uses it. They need the translated message to be presented clearly through text or audio so the interaction can continue naturally.
From Problems to Product Logic
Define
Defining the Right Product Direction
Build Around Accessible CommunicationWe shaped Kanvo around a simple principle: machine learning should stay behind the experience. Users should be able to communicate through familiar gestures while the product handles capture, recognition, translation, and output with as little friction as possible.
Let users communicate through gestures instead of forcing them to adapt their communication to a conventional text-based interface.
Transform recognized gestures into readable text and spoken audio that can be understood by someone who does not use sign language.
Design for cases where ML does not recognize a gesture, giving users a clear path to retry, edit, or work with their gesture Library.
Ideate
Turning Communication Barriers into Product Ideas
Rather than treating gesture recognition as an isolated AI feature, we designed supporting product flows around what happens before, during, and after recognition.
Making Everyday Communication More Accessible
Problem: Sign language users and people who do not understand sign language can face a direct communication barrier.Product logic: Capture gestures through the device camera, recognize supported signs, and transform the result into text and audio that the other person can understand.
Recognition Uncertainty
Problem: An ML model cannot be expected to understand every captured gesture correctly.Product logic: Give users control over the result through retaking, editing, confirmation, and flows for handling unrecognized gestures instead of turning recognition failure into a dead end.
Personal Communication Library
Problem: Users may repeatedly need gestures or phrases that the base recognition experience does not cover.Product logic: Introduce a personal Library where communication content can be organized into categories and new phrases can be added manually or through gesture capture.
Wireframes



Visual Design
Typography & Colors
Make Complex ML Feel Simple
to Use
Technology Behind the Experience, Not in the WayGesture recognition is technically complex, but Kanvo users should not have to understand how the technology works. We designed the experience around familiar actions - capture a gesture, review the result, make corrections when needed, and communicate.
Recognition states, clear controls, accessible typography, and straightforward navigation keep the focus on the conversation rather than the machine-learning process.



Designing Around Real Communication
Gesture Capture
Start With the User’s Natural InputWe designed the communication flow around gestures rather than asking users to adapt to a conventional text-first interface.
The camera experience keeps the interaction focused: users start capturing, perform their gesture, finish the recording, and move directly to the recognition result.



Turn Recognition Into an Actionable Message
Recognition does not end when the ML model returns a result. The output needs to become something another person can actually understand.
Kanvo presents the recognized message as text and supports audio playback through text-to-speech. Users can review the result, make corrections when necessary, confirm the message, and continue with another interaction.



Give Users a Way Forward
An unrecognized gesture should not become the end of the conversation. Kanvo includes recovery flows that let users retake the gesture or continue through the personal Library.
When recognition cannot identify a gesture, users can select or add communication content, organize it within a category, and capture the gesture again as part of the Library flow.



Organize Communication Around The User
The Library was designed as a structured space for communication content beyond the immediate recognition session.
Users can start with predefined categories or build their Library from scratch, browse stored content, and add new phrases either manually or through gesture capture. This gives the product a more personalized layer around the core recognition experience.



FAQ About Kanvo
What Is Kanvo?

Kanvo is an accessibility-focused communication application designed to help people with communication disabilities communicate through gestures. The product uses machine learning to process gestures captured through the device camera and translate recognized signs into text, with text-to-speech supporting communication with people who may not understand sign language.
What Problem Does Kanvo Solve?

Kanvo addresses communication barriers between people who use sign language and people who may not understand it. The product was designed to turn gesture-based communication into text and audio within an accessible digital experience.
How Does Kanvo Recognize Sign Language?

Kanvo uses a machine-learning recognition pipeline built around Apple’s Vision and AVFoundation technologies for camera processing, Core ML for recognition, and Create ML for model creation and training.
Was A Third-Party Sign Language Recognition API Used?

No ready-made API, SDK, or open-source solution identified during technical research could meet Kanvo’s requirements. This led the team to develop a custom recognition approach using Apple’s computer vision and machine-learning technologies.
How Does Gesture-Based Communication Work In Kanvo?

Users capture gestures through the device camera and the recognition system processes them into a text result. The communication flow also supports reviewing the result, text editing, audio playback, confirmation, and starting another communication interaction.
What Happens If Kanvo Does Not Recognize A Gesture?

The product includes recovery flows for unsuccessful recognition. Users can retake a gesture or continue through the Library experience, where communication content can be selected, organized, or added through manual input and gesture capture.
What Is The Kanvo Gesture Library?

The Library is a structured space for managing communication content. Users can work with categorized gestures and phrases, start with predefined categories or build their Library from scratch, and add new phrases manually or through gesture capture.
How Was Accessibility Addressed In Kanvo?

Accessibility was considered throughout the UX/UI process, including typography, color contrast, interaction patterns, and navigation. The design research considered WCAG 2.1 AA and Australian digital accessibility requirements.
What Platform Was Kanvo Developed For?

The Kanvo Alpha was developed as a native iPadOS application, allowing the product to combine camera-based gesture capture, Apple’s machine-learning technologies, and an interface designed around accessible communication.
What Technologies Were Used To Build Kanvo?

Kanvo’s technical ecosystem included native iPadOS development, Vision and AVFoundation for camera processing, Core ML and Create ML for machine learning, backend services, AWS cloud infrastructure, Firebase, and supporting monitoring and product services.
What Was You Are Launched Responsible For?

You are launched supported Kanvo across product discovery, business analysis, technical research, UX/UI design, native iPadOS development, ML research and integration, backend and infrastructure work, and web development for the supporting landing page.
What Was Delivered During The Kanvo Alpha Stage?

The Alpha established Kanvo’s core gesture-recognition experience within a native iPadOS application, connecting camera-based gesture capture, ML recognition, recognition results, text-to-speech, and supporting account and product flows within one working experience.
How Was Kanvo’s Recognition Model Improved?

During development, additional model training was introduced to improve recognition performance, including the model’s ability to distinguish a user’s hand across different backgrounds.
How Was Kanvo Designed To Evolve Beyond The Alpha?

Kanvo’s technical foundation was designed to support progressive recognition development. The roadmap extended from Alpha-stage image-based character recognition toward video-based word and gesture recognition and, ultimately, personalized on-device model training.
























