sigus.core
Class SigusEffect

java.lang.Object
  extended bysigus.core.SigusEffect
All Implemented Interfaces:
javax.media.Codec, javax.media.Controls, javax.media.Effect, javax.media.PlugIn
Direct Known Subclasses:
LibrasEffect, TTTEffect

public abstract class SigusEffect
extends java.lang.Object
implements javax.media.Effect

A SigusEffect integrates all the modules of a visual based system, which include some digital image processing filters, a feature extractor and a machine learning module.


Field Summary
(package private)  boolean calibrating
          True if system is in the calibration phase (e.g. extracting samples from human skin
(package private)  int focus
          The current identified class (e.g. the "signed" letter or the last square where the eyes looked at).
(package private)  int[] imageCopy
           
static ij.ImagePlus imp
          ImageJ window where the processed image will be presented
(package private) static ij.ImagePlus impCopy
          Auxiliary ImagePlus used when images must be saved on disk
(package private)  javax.media.Format inputFormat
          Format of the input image (e.g. 8-bit, RGB, HSV, etc)
(package private)  javax.media.Format[] inputFormats
          Supported input formats (the formats that the effect can process)
static ij.process.ImageProcessor ip
          ImageJ image processor.
(package private) static ij.process.ImageProcessor ipCopy
          Auxiliary ImageProcessor used when images must be saved on disk
(package private)  int lastFocus
          The last identified class (e.g. the last "signed" letter or the last square where the eyes looked at).
(package private)  java.awt.Rectangle lastRoi
          Coordinates of the last ROI - Region of Intereset
 Learner_ learner
          Algorithm that will be used to learn from examples
 int MAX_CLASS
          The number of different classes + 1 (e.g.: TTT has 9 classes - the 9 different screen positions.
 int maxFocus
          In order to be recognized, a sign must be detect in "maxFocus" consecutive frames.
 int minInstances
          The number of training instances that should be collected before the learning algorithm is called for the first time
(package private)  javax.media.Format outputFormat
          Format of the output image (e.g. 8-bit, RGB, HSV, etc)
(package private)  javax.media.Format[] outputFormats
          Support output formats (the formats that the effect can generate)
 ParameterExtractor_ pe
          Extracts the parameters from a segmented and ROI marked image
(package private)  int sameFocusCounter
          Counter for the number of times that the same sign has been recognizedThe current identified class (e.g. the "signed" letter or the last square where the eyes looked at).
 boolean saveInstancesImages
          True if the frames captured (before and after processing) should be saved on disk
 SigusApp sigusApp
          Application that will process the recognized visual signs
 Updater updater
          The updater reads each training sample and writes it, converts it to arff (weka) format, and writes it to an arff file
 
Fields inherited from interface javax.media.PlugIn
BUFFER_PROCESSED_FAILED, BUFFER_PROCESSED_OK, INPUT_BUFFER_NOT_CONSUMED, OUTPUT_BUFFER_NOT_FILLED, PLUGIN_TERMINATED
 
Constructor Summary
SigusEffect()
          Set focus related variables and supported formats
 
Method Summary
abstract  void applyFilters(byte[] inData, byte[] jmfData, byte[] outData, int width, int height)
          The Digital Image Processing algorithms should be implemented or started in this function
 void close()
           
(package private)  void copyImage(byte[] inData, int[] imageCopy, int height, int width)
          Copy the image to the imageCopy vector
abstract  void createParameterExtractor()
           
(package private)  void createProcessedImageWindow(int width, int height)
          Create the window where the processed image is shown
abstract  void createSigusApp()
           
abstract  void createUpdater()
           
 java.lang.Object getControl(java.lang.String controlType)
           
 java.lang.Object[] getControls()
           
 java.lang.String getName()
           
 javax.media.Format[] getSupportedInputFormats()
           
 javax.media.Format[] getSupportedOutputFormats(javax.media.Format input)
          Get the supported output formats.
abstract  void init()
           
 boolean isCalibrating()
           
(package private)  javax.media.Format matches(javax.media.Format in, javax.media.Format[] outs)
          Test if a specific format is compatible to any format from the list of supported formats
 void open()
           
 int process(javax.media.Buffer inBuffer, javax.media.Buffer outBuffer)
          Takes a input image, inBuffer, and produces a processed image,outBuffer.
 void reset()
           
 void resetLearner()
           
(package private)  void saveImage(byte[] inData, byte[] outData, int width, int height)
          Save two images to disk: before and after processing
 javax.media.Format setInputFormat(javax.media.Format input)
           
 javax.media.Format setOutputFormat(javax.media.Format output)
           
 void setSaveInstances(boolean value)
          Start or stop saving images to disk
(package private)  byte[] validateByteArraySize(javax.media.Buffer buffer, int newSize)
          Verify if buffer can be properly converted to a byte array.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

inputFormat

javax.media.Format inputFormat
Format of the input image (e.g. 8-bit, RGB, HSV, etc)


outputFormat

javax.media.Format outputFormat
Format of the output image (e.g. 8-bit, RGB, HSV, etc)


inputFormats

javax.media.Format[] inputFormats
Supported input formats (the formats that the effect can process)


outputFormats

javax.media.Format[] outputFormats
Support output formats (the formats that the effect can generate)


sigusApp

public SigusApp sigusApp
Application that will process the recognized visual signs


learner

public Learner_ learner
Algorithm that will be used to learn from examples


updater

public Updater updater
The updater reads each training sample and writes it, converts it to arff (weka) format, and writes it to an arff file


pe

public ParameterExtractor_ pe
Extracts the parameters from a segmented and ROI marked image


MAX_CLASS

public int MAX_CLASS
The number of different classes + 1 (e.g.: TTT has 9 classes - the 9 different screen positions. The LIBRAS Prototype has 36 classes: all 26 lower case letters plus 10 digits. The aditional class is used to indicate "no class detected yet"


lastFocus

int lastFocus
The last identified class (e.g. the last "signed" letter or the last square where the eyes looked at).


focus

int focus
The current identified class (e.g. the "signed" letter or the last square where the eyes looked at).


sameFocusCounter

int sameFocusCounter
Counter for the number of times that the same sign has been recognizedThe current identified class (e.g. the "signed" letter or the last square where the eyes looked at).


maxFocus

public int maxFocus
In order to be recognized, a sign must be detect in "maxFocus" consecutive frames.


minInstances

public int minInstances
The number of training instances that should be collected before the learning algorithm is called for the first time


saveInstancesImages

public boolean saveInstancesImages
True if the frames captured (before and after processing) should be saved on disk


calibrating

boolean calibrating
True if system is in the calibration phase (e.g. extracting samples from human skin


ip

public static ij.process.ImageProcessor ip
ImageJ image processor. Unfortunately, we didn't find a way to work directly in the JMF Image Window using ImageJ tools. That is why a second Image Window is created and pointed by ip and imp. It is in this second window most part of the image processing happens


imp

public static ij.ImagePlus imp
ImageJ window where the processed image will be presented


imageCopy

int[] imageCopy

ipCopy

static ij.process.ImageProcessor ipCopy
Auxiliary ImageProcessor used when images must be saved on disk


impCopy

static ij.ImagePlus impCopy
Auxiliary ImagePlus used when images must be saved on disk


lastRoi

java.awt.Rectangle lastRoi
Coordinates of the last ROI - Region of Intereset

Constructor Detail

SigusEffect

public SigusEffect()
Set focus related variables and supported formats

Method Detail

process

public int process(javax.media.Buffer inBuffer,
                   javax.media.Buffer outBuffer)
Takes a input image, inBuffer, and produces a processed image,outBuffer. Control the learning process.

Specified by:
process in interface javax.media.Codec
Parameters:
inBuffer - Buffer containing image captured by CameraReader.class
outBuffer - Buffer that will be returned to CameraReader.class
Returns:
Status of the process (ok or not ok).

setSaveInstances

public void setSaveInstances(boolean value)
Start or stop saving images to disk


createProcessedImageWindow

void createProcessedImageWindow(int width,
                                int height)
Create the window where the processed image is shown


copyImage

void copyImage(byte[] inData,
               int[] imageCopy,
               int height,
               int width)
Copy the image to the imageCopy vector

Parameters:
width - Image width
height - Image height

saveImage

void saveImage(byte[] inData,
               byte[] outData,
               int width,
               int height)
Save two images to disk: before and after processing

Parameters:
width - Image width
height - Image height

isCalibrating

public boolean isCalibrating()

resetLearner

public void resetLearner()

applyFilters

public abstract void applyFilters(byte[] inData,
                                  byte[] jmfData,
                                  byte[] outData,
                                  int width,
                                  int height)
The Digital Image Processing algorithms should be implemented or started in this function


createParameterExtractor

public abstract void createParameterExtractor()

createUpdater

public abstract void createUpdater()

createSigusApp

public abstract void createSigusApp()

init

public abstract void init()

getName

public java.lang.String getName()
Specified by:
getName in interface javax.media.PlugIn

open

public void open()
Specified by:
open in interface javax.media.PlugIn

close

public void close()
Specified by:
close in interface javax.media.PlugIn

reset

public void reset()
Specified by:
reset in interface javax.media.PlugIn

getControl

public java.lang.Object getControl(java.lang.String controlType)
Specified by:
getControl in interface javax.media.Controls

getControls

public java.lang.Object[] getControls()
Specified by:
getControls in interface javax.media.Controls

getSupportedInputFormats

public javax.media.Format[] getSupportedInputFormats()
Specified by:
getSupportedInputFormats in interface javax.media.Codec

getSupportedOutputFormats

public javax.media.Format[] getSupportedOutputFormats(javax.media.Format input)
Get the supported output formats.

Specified by:
getSupportedOutputFormats in interface javax.media.Codec
Parameters:
input - If input is not null, return only the output formats that are compatible to the input format.
Returns:
Supported formats

setInputFormat

public javax.media.Format setInputFormat(javax.media.Format input)
Specified by:
setInputFormat in interface javax.media.Codec

setOutputFormat

public javax.media.Format setOutputFormat(javax.media.Format output)
Specified by:
setOutputFormat in interface javax.media.Codec

matches

javax.media.Format matches(javax.media.Format in,
                           javax.media.Format[] outs)
Test if a specific format is compatible to any format from the list of supported formats

Parameters:
in - format to be tested
outs - list of supported formats
Returns:
supported format compatible to in

validateByteArraySize

byte[] validateByteArraySize(javax.media.Buffer buffer,
                             int newSize)
Verify if buffer can be properly converted to a byte array.

Parameters:
buffer - buffer containing the image.
newSize - the size of the byte array
Returns:
image converted to a byte array.