It's a common misconception among app developers that incorporating machine learning (ML) into any app is difficult. It's a lot easier to incorporate machine learning into your apps now, thanks to recent advances in ML technologies, even if you don't have any prior experience in the subject.
With platforms and development tools like Fritz AI and Firebase's ML Kit, incorporating machine learning into your mobile app development toolset is becoming simpler. By packaging pre-trained machine learning models for usage, these tools hide technological complexity. You may also train and import your own models with them. So, in this blog, we will be exploring how to perform text recognition in Flutter using Firebase’s ML Kit.
Google's ML Kit SDK is a relatively new product that was introduced in 2018. ML Kit is a software development kit that makes it easier for developers to include machine learning models into mobile apps. In addition, even a junior developer may complete this task. The ML Kit mobile SDK is a powerful yet simple-to-use package that brings Google's machine learning capabilities to Android and iOS apps. You can obtain the functionality you need with only a few lines of code, whether you're new to machine learning or a seasoned veteran. You don't need much expertise with neural networks or model optimization to get started.
ML Kit makes it easy to apply machine learner methods on your applications by integrating Google's ML technology, including the Google Cloud Vision API, TensorFlow Lite, and Android Neural Networks API in a single SDK. Just a few lines of code allow you to obtain the power of cloud-based processing, mobile on-device optimized models or the flexibility of customized TensorFlow Lite models in real-time.
Here are some benefits of using Firebase ML Kit:
ML Kit provides a collection of ready-to-use APIs for typical mobile use cases, including text recognition, face detection, landmark identification, barcode scanning, picture labelling, and text language identification. Simply input data into the ML Kit library, and it will provide you with the information you want.
The APIs in the ML Kit operates on-device or in the cloud. Its on-device APIs can process your data rapidly and function even if you don't have access to the internet. And cloud-based APIs, on the other hand, take advantage of Google Cloud's machine learning technologies to improve accuracy even more.
Read our other post on What’s New in Flutter 2.2.0 to make it more adaptive?
If the APIs provided by ML Kit do not meet your needs, you may always introduce your own TensorFlow Lite models. Simply upload your model to Firebase, and it’ll host and provide it to your app for you. ML Kit works with your own model as an API layer, making it easier to execute and utilize.
To make use of all Firebase Machine Learning has to offer, you must first create a Firebase project in the Firebase console. So, create a Firebase Project, connect it with Flutter, and simultaneously add a dependency for Flutter ML Kit in Firebase. As we move past this, we will use the ML Kit addition in the Firebase to recognize text.
First, you need to create a new Firebase Project on either the Android or iOS Platform.
Click on Add Project to create the new project and enter the project name. Now, Firebase Console will create a new project in the Dashboard.
We will be building a Flutter app containing two different screens, i.e., CameraScreen (to show camera view and click pictures) and a DetailScreen (to show recognized text details from the image).
To let your Flutter app use the device camera, you need to add a separate plugin to the app. This will allow for retrieval of all camera data available as well as access to it.
Add plugin to your pubspec.yaml file:
camera: ^0.5.7+4
Replace the demo counter app code available in main.dart file with the code given here:
import 'package:flutter/material.dart';
import 'package:camera/camera.dart';
// Global variable for storing the list of
// cameras available
List<CameraDescription> cameras = [];
Future<void> main() async {
try {
WidgetsFlutterBinding.ensureInitialized();
// Retrieve the device cameras
cameras = await availableCameras();
} on CameraException catch (e) {
print(e);
}
runApp(MyApp());
}
class MyApp extends StatelessWidget {
@override
Widget build(BuildContext context) {
return MaterialApp(
title: 'ML Vision',
theme: ThemeData(
primarySwatch: Colors.blue,
),
home: CameraScreen(),
);
}
}
Now, define CameraScreen that will let you see the button for clicking images and see the camera preview.
class CameraScreen extends StatefulWidget {
@override
_CameraScreenState createState() => _CameraScreenState();
}
class _CameraScreenState extends State<CameraScreen> {
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(
title: Text('ML Vision'),
),
body: Container(),
);
}
}
// Inside _CameraScreenState class
CameraController _controller;
Check out here how to create Video-Streaming App with Flutter (Using MUX)
@override
void initState() {
super.initState();
_controller = CameraController(cameras[0], ResolutionPreset.medium);
_controller.initialize().then((_) {
if (!mounted) {
return;
}
setState(() {});
});
}
There are two parameters, which are required by CameraController()
@override
void dispose() {
_controller.dispose();
super.dispose();
}
path_provider: ^1.6.7
intl: ^0.16.1
import 'dart:io';
import 'package:intl/intl.dart';
import 'package:path_provider/path_provider.dart';
// ...
Future<String> _takePicture() async {
// Checking whether the controller is initialized
if (!_controller.value.isInitialized) {
print("Controller is not initialized");
return null;
}
// Formatting Date and Time
String dateTime = DateFormat.yMMMd()
.addPattern('-')
.add_Hms()
.format(DateTime.now())
.toString();
String formattedDateTime = dateTime.replaceAll(' ', '');
print("Formatted: $formattedDateTime");
// Retrieving the path for saving an image
final Directory appDocDir = await getApplicationDocumentsDirectory();
final String visionDir = '${appDocDir.path}/Photos/Vision\ Images';
await Directory(visionDir).create(recursive: true);
final String imagePath = '$visionDir/image_$formattedDateTime.jpg';
// Checking whether the picture is being taken
// to prevent execution of the function again
// if previous execution has not ended
if (_controller.value.isTakingPicture) {
print("Processing is in progress...");
return null;
}
try {
// Captures the image and saves it to the
// provided path
await _controller.takePicture(imagePath);
} on CameraException catch (e) {
print("Camera Exception: $e");
return null;
}
return imagePath;
}
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(
title: Text('ML Vision'),
),
body: _controller.value.isInitialized
? Stack(
children: <Widget>[
CameraPreview(_controller),
Padding(
padding: const EdgeInsets.all(20.0),
child: Container(
alignment: Alignment.bottomCenter,
child: RaisedButton.icon(
icon: Icon(Icons.camera),
label: Text("Click"),
onPressed: () async {
await _takePicture().then((String path) {
if (path != null) {
Navigator.push(
context,
MaterialPageRoute(
builder: (context) => DetailScreen(path),
),
);
}
});
},
),
),
)
],
)
: Container(
color: Colors.black,
child: Center(
child: CircularProgressIndicator(),
),
),
);
}
As in the previous step, the camera is added to your app. So, now you have to analyze the input given by the camera, and that’s all this is about. So, import the Firebase ML Vision library to your app and define the image retrieval size and text recognition parameters.
firebase_ml_vision: ^0.9.3+8
// Inside image_detail.dart file
Now, you will have to pass the path of the image file to the DetailScreen. here is the DetailScreen basic structure:
import 'package:flutter/material.dart';
import 'package:firebase_ml_vision/firebase_ml_vision.dart';
import 'dart:io';
import 'dart:ui';
import 'dart:async';
class DetailScreen extends StatefulWidget {
final String imagePath;
DetailScreen(this.imagePath);
@override
_DetailScreenState createState() => new _DetailScreenState(imagePath);
}
class _DetailScreenState extends State<DetailScreen> {
_DetailScreenState(this.path);
final String path;
Size _imageSize;
String recognizedText = "Loading ...";
void _initializeVision() async {
// TODO: Initialize the text recognizer here
}
Future<void> _getImageSize(File imageFile) async {
// TODO: Retrieve the image size here
}
@override
void initState() {
_initializeVision();
super.initState();
}
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(
title: Text("Image Details"),
),
body: Container(),
);
}
}
As you would have completed the definition of the CameraScreen, we will now work on building the UI for DetailScreen. So, for this, you need to stack two different widgets into your app, i.e., one for displaying the image and the other for showing the text extracted.
override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(
title: Text("Image Details"),
),
body: _imageSize != null
? Stack(
children: <Widget>[
Center(
child: Container(
width: double.maxFinite,
color: Colors.black,
child: AspectRatio(
aspectRatio: _imageSize.aspectRatio,
child: Image.file(
File(path),
),
),
),
),
Align(
alignment: Alignment.bottomCenter,
child: Card(
elevation: 8,
color: Colors.white,
child: Padding(
padding: const EdgeInsets.all(16.0),
child: Column(
mainAxisSize: MainAxisSize.min,
crossAxisAlignment: CrossAxisAlignment.start,
children: <Widget>[
Row(),
Padding(
padding: const EdgeInsets.only(bottom: 8.0),
child: Text(
"Identified emails",
style: TextStyle(
fontSize: 20,
fontWeight: FontWeight.bold,
),
),
),
Container(
height: 60,
child: SingleChildScrollView(
child: Text(
recognizedText,
),
),
),
],
),
),
),
),
],
)
: Container(
color: Colors.black,
child: Center(
child: CircularProgressIndicator(),
),
),
);
}
As a final step, you have already configured the main parts of your application, i.e., analyzing pictures and recognizing text. Now, an optional step allows you to mark the text you captured in the image itself. For this, you need to add a CustomPaint object into your app to mark the images.
The final step is to get your app running and configure it correctly according to the specific platform. And, with that, your app is ready to function on the device of your choice.
For this, go to the project directory - Android - App - build.gradle and set the minSDKVersion to 21.
minSdkVersion 21
iOS
Now, you will have to add the following in ios/Runner/Info.plist:
<key>NSCameraUsageDescription</key>
<string>Can I use the camera please?</string>
<key>NSMicrophoneUsageDescription</key>
<string>Can I use the mic please?</string>
For every step involved, add the specific methods to the path and ensure that Firebase ML Vision models are adequately defined.
Using Firebase's ML Kit, you can see how quick and straightforward it is to implement machine learning into your mobile apps. Before putting this app into production, double-check that you have all of the necessary configurations in place using Firebase's checklist. If you decide to utilize the cloud-based API, make sure you have the appropriate API access privileges. This will prevent your application from failing in production.
At your Team in India, we have a team of flutter experts. If you want to hire Flutter developers or have any questions on what all services we offer at Your team in India– Click here to contact us.