SharpSenses

SharpSenses

Andre Carlucci

Andre Carlucci

Saco Grande, Santa Catarina

An easier way to use Intel 3d Cameras. Custom poses, gestures and much more.

Intel RealSense™

Description

An easier way to use the RealSense SDK! Custom poses, gestures and much more.

Warning: Make sure you have the RealSense SDK version 10 installed before using SharpSenses

How to use:

Nuget: Install-Package SharpSenses.RealSense

Sample:

var cam = Camera.Create(Capability.HandTracking);
cam.LeftHand.Visible += (s,a) => Console.WriteLine("Hi left hand!");
cam.RightHand.Closed += (s,a) => Console.WriteLine("Hand Closed");
cam.RightHand.Moved += (s,a) => {
    Console.WriteLine("-> x:{0} y:{1}", a.Position.Image.X, a.Position.Image.Y);
}
cam.Start();

Enabling Capabilities

For performance reasons, you have to tell the camera which modules will be loaded for use. The available modules are:

- HandTracking,
- FingersTracking,
- GestureTracking,
- FaceTracking,
- FaceRecognition,
- FacialExpressionTracking,
- ImageStreamTracking,
- SegmentationStreamTracking

You can enable the modules when creating the Camera object or calling the method "AddCapability", always before calling "Start".

var cam = Camera.Create(Capability.HandTracking, Capability.FingersTracking);
or
cam.AddCapability(Capability.FaceTracking);

Examples:

Gestures:

cam.Gestures.SlideLeft += (s, a) => Console.WriteLine("Swipe Left");
cam.Gestures.SlideRight += (s, a) => Console.WriteLine("Swipe Right");
cam.Gestures.SlideUp += (s, a) => Console.WriteLine("Swipe Up");
cam.Gestures.SlideDown += (s, a) => Console.WriteLine("Swipe Down");

Poses:

cam.Poses.PeaceBegin += (s, a) => Console.WriteLine("Make love, not war");
cam.Poses.PeaceEnd += (s, a) => Console.WriteLine("Bye!");

Eyes:

cam.Face.LeftEye.Blink += (sender, eventArgs) => {
    Console.WriteLine("Blink");
};
cam.Face.LeftEye.DoubleBlink += (sender, eventArgs) => {
    Console.WriteLine("Double Blink");
};
cam.Face.WinkedLeft += (sender, eventArgs) => {
    Console.WriteLine("WinkedLeft");
};
cam.Face.WinkedRight += (sender, eventArgs) => {
    Console.WriteLine("WinkedRight");
};

Mouth:

cam.Face.Mouth.Opened += (s, a) => {
    Console.WriteLine("-> Mouth opened");
};

cam.Face.Mouth.Closed += (s, a) => {
    Console.WriteLine("-> Mouth closed");
};

cam.Face.Mouth.Smiled += (s, a) => {
    Console.WriteLine("-> Mouth smiled");
};

Custom Poses:

var pose = PoseBuilder.Create().ShouldBeNear(_cam.LeftHand, _cam.RightHand, 100).Build();
    pose.Begin += (s, a) => {
    Console.WriteLine("Super pose!");
};
pose.Begin += (s, a) => DoSomething();

Facial Expressions:

- Anger
- Contempt
- Disgust
- Fear
- Joy
- Sadness
- Surprise

OBS: Unfortunately this feature was deprecated by Intel

Face Recognition:

Anytime you want to recognite a new face, call:

cam.Face.RecognizeFace();

You can always get a notification when a new or pre-recognized face is recognized:

_cam.Face.PersonRecognized += (s, a) => {
    Console.WriteLine("Hello " + a.UserId); 
};

Voice/Speech Synthesis:

Oh yeah, we speak!

cam.Speech.Say("Isn't that cool?");

Voice/Speech Recognition:

I can hear you, man!

cam.Speech.SpeechRecognized += (s, a) => {
    Console.WriteLine("-> " + a.Sentence);
};
cam.Speech.EnableRecognition();

Don't forget that you have to have the Intel RealSense SDK v10 (and the 3d camera, of course) for this library to work!

Links

Standard github

Medium andrecarlucci

Andre C. created project SharpSenses

Medium 5563118b c45e 4459 a711 22fb8bb3d6db

SharpSenses

An easier way to use the RealSense SDK! Custom poses, gestures and much more.

Warning: Make sure you have the RealSense SDK version 10 installed before using SharpSenses

How to use:

Nuget: Install-Package SharpSenses.RealSense

Sample:

var cam = Camera.Create(Capability.HandTracking);
cam.LeftHand.Visible += (s,a) => Console.WriteLine("Hi left hand!");
cam.RightHand.Closed += (s,a) => Console.WriteLine("Hand Closed");
cam.RightHand.Moved += (s,a) => {
    Console.WriteLine("-> x:{0} y:{1}", a.Position.Image.X, a.Position.Image.Y);
}
cam.Start();

Enabling Capabilities

For performance reasons, you have to tell the camera which modules will be loaded for use. The available modules are:

- HandTracking,
- FingersTracking,
- GestureTracking,
- FaceTracking,
- FaceRecognition,
- FacialExpressionTracking,
- ImageStreamTracking,
- SegmentationStreamTracking

You can enable the modules when creating the Camera object or calling the method "AddCapability", always before calling "Start".

var cam = Camera.Create(Capability.HandTracking, Capability.FingersTracking);
or
cam.AddCapability(Capability.FaceTracking);

Examples:

Gestures:

cam.Gestures.SlideLeft += (s, a) => Console.WriteLine("Swipe Left");
cam.Gestures.SlideRight += (s, a) => Console.WriteLine("Swipe Right");
cam.Gestures.SlideUp += (s, a) => Console.WriteLine("Swipe Up");
cam.Gestures.SlideDown += (s, a) => Console.WriteLine("Swipe Down");

Poses:

cam.Poses.PeaceBegin += (s, a) => Console.WriteLine("Make love, not war");
cam.Poses.PeaceEnd += (s, a) => Console.WriteLine("Bye!");

Eyes:

cam.Face.LeftEye.Blink += (sender, eventArgs) => {
    Console.WriteLine("Blink");
};
cam.Face.LeftEye.DoubleBlink += (sender, eventArgs) => {
    Console.WriteLine("Double Blink");
};
cam.Face.WinkedLeft += (sender, eventArgs) => {
    Console.WriteLine("WinkedLeft");
};
cam.Face.WinkedRight += (sender, eventArgs) => {
    Console.WriteLine("WinkedRight");
};

Mouth:

cam.Face.Mouth.Opened += (s, a) => {
    Console.WriteLine("-> Mouth opened");
};

cam.Face.Mouth.Closed += (s, a) => {
    Console.WriteLine("-> Mouth closed");
};

cam.Face.Mouth.Smiled += (s, a) => {
    Console.WriteLine("-> Mouth smiled");
};

Custom Poses:

var pose = PoseBuilder.Create().ShouldBeNear(_cam.LeftHand, _cam.RightHand, 100).Build();
    pose.Begin += (s, a) => {
    Console.WriteLine("Super pose!");
};
pose.Begin += (s, a) => DoSomething();

Facial Expressions:

- Anger
- Contempt
- Disgust
- Fear
- Joy
- Sadness
- Surprise

OBS: Unfortunately this feature was deprecated by Intel

Face Recognition:

Anytime you want to recognite a new face, call:

cam.Face.RecognizeFace();

You can always get a notification when a new or pre-recognized face is recognized:

_cam.Face.PersonRecognized += (s, a) => {
    Console.WriteLine("Hello " + a.UserId); 
};

Voice/Speech Synthesis:

Oh yeah, we speak!

cam.Speech.Say("Isn't that cool?");

Voice/Speech Recognition:

I can hear you, man!

cam.Speech.SpeechRecognized += (s, a) => {
    Console.WriteLine("-> " + a.Sentence);
};
cam.Speech.EnableRecognition();

Don't forget that you have to have the Intel RealSense SDK v10 (and the 3d camera, of course) for this library to work!

Thumb andrecarlucci

Andre C.

Version 0.9.7 on Nuget :) - Compatibility with R5 - More performance using Capabilities - 3D Segmentation added (Thanks to Felipe Pedroso)

No users to show at the moment.

Default user avatar 57012e2942
  • Projects 0
  • Followers 0

awadhi ally

Dar es Salaam, Tanzania

Bigger 20161112 172934
  • Projects 0
  • Followers 5

Dennis Masesi

A passionate student with the love of Artificial Intelligence

Kenya

Bigger 14264126 1719610614966859 7658205800800384838 n
  • Projects 0
  • Followers 6

Caleb Pfohl

151 New Park Ave # 75, Hartford, CT 06106, USA

Bigger bob duffy 3d head avatar
  • Projects 4
  • Followers 130

Bob Duffy

Folsom, CA, USA

Bigger img 20170629 212404
  • Projects 1
  • Followers 19

Edwin Maye

Web Apps | Google Developer Groups Lead | O365 & MS Azure | Python Beginner | Blogger | #MeetMaye | Key Learnings® | Ugali + Fish = Me | Graphics & Design 👌

Nairobi, Kenya

Bigger j5oui22x 400x400
  • Projects 0
  • Followers 2

Thomas Jung

engineer, hacker, maker, federal hackathon winner.

Dallas, TX, USA

Bigger aaeaaqaaaaaaaaktaaaajdc4nzniztrmlwu1nmitnde4ni1iytrilwu4zmizymmwzdq1yq
  • Projects 6
  • Followers 59

Paulo Gurgel Pinheiro

Co-founder and CEO of HOOBOX Robotics, the startup behind the next generation of interfaces for monitoring people behaviors. Ph.D. in Computer Science, Paulo is developer recognized by the Intel® Software Innovator program.

Campinas - State of São Paulo, Brazil

See More