Visual Perceptive Computing

The ever increasing capabilities of consumer sensors are reshaping the way we interact with computing devices, making it more natural, intuitive and immersive. Devices are able to perceive actions through gestures, finger articulation, body- or face tracking and more. The field of capturing and interpreting these extensive data streams, coined perceptive computing, is poised to profoundly alter medical and health applications and allow quantitative everywhere-assessment of clinical symptoms previously restricted to the physician’s subjective expertise.

At the core of these advances are sensors that three-dimensionally scan their environment and are able to accurately measure shapes and movements in the space in front of them. With current 3D cameras like the Microsoft® KinectTM or Intel® RealSenseTM body parts can be tracked with an accuracy in the millimetre range - without requiring the user to wear any markers or special clothing.

Motion Analysis Technology from Motognosis

We utilize visual perceptive computing to identify motor dysfunction in neurologic disorders.

To achieve this we are developing specific assessements optimized for different neurologic conditions. Depending on the assessment and setting, the algorithmic techniques required to identify and compute diagnostic parameters vary widely. Generally speaking, we utilize motion analysis, the field of capturing and processing motion in videos, leveraging volumetric data and combining it with other sensor streams, such as audio or RGB video. A typical workflow is as follows:

  1. The subject to be tested is performing a sequence of movements that was developed to uncover specific motor dysfunction.
  2. During the test all 3D data is recorded and then further analyzed. One integral part of the Motognosis analysis toolset is skeletal tracking, where raw video data is algorithmically mapped to a model of the body. For assessments where generic tracking algorithms fail, we have developed specifically trained segmentation algorithms, extracting exactly the movement signature of diagnostic value.
  3. From the change of position and orientation of body parts, we derive specific movement signals, which in turn are processed using signal analysis techniques. The resulting, assessment specific outcome parameters are robust quantifications of the important aspects of motor impairment. We additionally provide analysis modules directly quantifying depth streams and using computer vision approaches to generate aggregate visualizations of depth videos, of which motion profiles are one example, that can be used for validation of recordings or exploratory evaluation of movement patterns.

We currently use the Microsoft Kinect 2 sensor for our running systems. Our modular technology can be adapted to different hardware and software platforms. If you are interested in using our technology in your product,

Please contact us.

References

Technology from Motognosis and the Motognosis Labs software have been used and are currently being used in several projects.

Peer reviewed papers:

  • Rasche, L., Scheel, M., Otte, K., Althoff, P., van Vuuren, A.B., Gieß, R.M., Kuchling, J., Bellmann-Strobl, J., Ruprecht, K., Paul, F., Brandt, A.U., Schmitz-Hübsch, T., 2018. MRI Markers and Functional Performance in Patients With CIS and MS: A Cross-Sectional Study. Frontiers in Neurology 9.
  • Krüger, T., Behrens, J.R., Grobelny, A., Otte, K., Mansow-Model, S., Kayser, B., Bellmann-Strobl, J., Brandt, A.U., Paul, F., Schmitz-Hübsch, T., 2017. Subjective and objective assessment of physical activity in multiple sclerosis and their relation to health-related quality of life. BMC Neurology 17.
  • Grobelny, A., Behrens, J.R., Mertens, S., Otte, K., Mansow-Model, S., Krüger, T., Gusho, E., Bellmann-Strobl, J., Paul, F., Brandt, A.U., Schmitz-Hübsch, T., 2017. Maximum walking speed in multiple sclerosis assessed with visual perceptive computing. PLOS ONE 12.
  • Otte, K., Kayser, B., Mansow-Model, S., Verrel, J., Paul, F., Brandt, A.U., Schmitz-Hübsch, T., 2016. Accuracy and reliability of the kinect version 2 for clinical measurement of motor function. PloS one 11.
  • Ellermeyer, T., Otte, K., Heinrich, F., Mansow-Model, S., Kayser, B., Lipp, A., Seidel, A., Krause, P., Lauritsch, K., Gusho, E., Paul, F., Kühn, A.A., Brandt, A.U., Schmitz-Hübsch, T., 2016. Ranking of Dystonia Severity by Pairwise Video Comparison. Movement Disorders Clinical Practice 3, 587–595.
  • Behrens, J.R., Mertens, S., Krüger, T., Grobelny, A., Otte, K., Mansow-Model, S., Gusho, E., Paul, F., Brandt, A.U., Schmitz-Hübsch, T., 2016. Validity of visual perceptive computing for static posturography in patients with multiple sclerosis. Multiple Sclerosis Journal 22, 1596–1606.
  • Behrens, J., Pfüller, C., Mansow-Model, S., Otte, K., Paul, F., Brandt, A.U., 2014. Using perceptive computing in multiple sclerosis - the Short Maximum Speed Walk test. Journal of NeuroEngineering and Rehabilitation 11, 89.
Conference posters and talks:
  • Otte, K., Heinrich, F., Ellermeyer, T., Kayser, B., Mansow-Model, S., Paul, F., Brandt, A.U., Skowronek, C., Lipp, A., Schmitz-Hübsch, T., 2018. Evaluation of visual perceptive computing for Tremor Analysis. Mov Disord. 33 (suppl 2), 1.
  • Otte, K., Rasche, L., Röhling, H., Kayser, B., Mansow-Model, S., Paul, F., Brandt, A.U., Lipp, A., Schmitz-Hübsch, T., 2018. Instrumental assessment of upper limb bradykinesia in video-augmented UPDRS testing. Mov Disord. 33 (suppl 2), 1.
  • Otte, K., Rasche, L., Mansow-Model, S., Kayser, B., Gusho, E., Bellmann-Strobl, J., Paul, F., Schmitz-Hbsch, T., Brandt, A.U., 2017. A Kinect-based perceptive assessment battery for motor dysfunction in multiple sclerosis and other neuroinflammatory disorders. Mult Scler J, pp. 473–474.
  • Otte, Karen, Vater, T., Ellermeyer, T., Rasche, L., Wenzel, G., Kayser, B., Mansow-Model, S., Kuehn, A., Paul, F., Brandt, A., 2017. Instrumental Measurement of Stepping in Place - Detection of Asymmetry and Freezing of Gait, Mov Disord. 32 (suppl 2)
  • Mansow-Model, S., Schmitz-Hübsch, T., Otte, K., Rasche, L., Vater, T.-S., Krüger, T., Kayser, B., Kühn, A., Paul, F., Lipp, A., Brandt, A.U., 2017. PASS-PD: A clinically feasible measurement protocol to assess PD motor symptoms with visuoperceptive computing, in: Basal Ganglia. p. 1.
  • Otte, K., Kayser, B., Mansow-Model, S., Brandt, A.U., Verrel, J., Paul, F., Schmitz-Hübsch, Tanja, 2016. Evaluation of kinemtic parameters of potential clinical use extracted from Microsoft Kinect V2 motor assessments. Presented at the Movement Disorders.
  • Otte, K., Kayser, B., Mansow-Model, S., Brandt, A.U., Verrel, J., Schmitz-Huebsch, T., 2016. Spatial accuracy and reliability of Microsoft Kinect V2 in the assessment of joint movement in comparison to marker-based motion capture (Vicon), in: Movement Disorders. WILEY-BLACKWELL 111 RIVER ST, HOBOKEN 07030-5774, NJ USA, pp. S188–S189.
  • Otte, K., Kayser, Grobelny, A., Krüger, T., Mansow-Model, S., Brandt, A.U., Paul, F., Schmitz-Hübsch, Tanja, 2016. An accurate and portable System for measuring objective clinical Parameters in Patients with Multiple Sclerosis. Presented at the Human Motion Analysis for Healthcare Applications - an IET event.
  • Brandt, A., Behrens, J., Otte, K., Mansow-Model, S., Gusho, E., Mertens, S., Grobelny, A., Krüger, T., Paul, F., 2015. Development of a comprehensive motor function assessment battery for multiple sclerosis using perceptive computing (P3. 223), in: Neurology. pp. P3–223.
  • Mansow-Model, S., Otte, K., Krüger, Theresa, Schmitz-Hübsch, Tanja, 2017. PASS-MS – a clinically feasible measurement protocol to assess MS motor symptoms with visuo-perceptive computing. Presented at the 16. Deutscher Kongress für Versorgungsforschung (DKVF), German Medical Science GMS Publishing House, Düsseldorf.
  • Behrens, J., Otte, K., Mansow-Model, S., Brandt, A., Paul, F., 2014. Kinect-Based Gait Analysis In Patients With Multiple Sclerosis (P3. 135), in: Neurology. pp. P3–135.
  • Behrens JR, Gusho E, Mertens S, Otte K, Mansow-Model S, Paul F, et al. Postural control analysis in multiple sclerosis with perceptive computing based on Microsoft’s Kinect. Mult Scler J. 2014 Sep;20:61–61.
  • Behrens JR, Otte K, Mansow-Model S, Brandt AU, Paul F. Microsoft Kinect-based gait analysis in multiple sclerosis patients. Mult Scler J. 2013 Oct;19(11):263–263.
  • Pfueller, C., Otte, K., Mansow-Model, S., Paul, F., Brandt, A., 2013. Kinect-Based Analysis of Posture, Gait and Coordination in Multiple Sclerosis Patients. Neurology 80, P04.097.