Skip to main content

Getting Started

Install Dependencies​

Install Dependencies:

Build the Demo App​

  1. Download and unzip the Rogue Vision SDK bundle. This is a zip file provided to you by Rogue.
unzip rogue-vision-sdk-latest.zip
  1. Install mobile app dependencies
cd rogue-vision-react-native-demo
yarn
npx pod-install
  1. Build for iOS
yarn start # starts React Native / Metro
yarn run ios
  1. Build for Android
yarn start # starts React Native / Metro
yarn run android
note

Make sure your Node binary is accesible to xcode and your shell.

BEFORE YOU RUN IN IOS:

echo "export NODE_BINARY=$(which node)" >> rogue-vision-demo/ios/.xcode.env

ON MAC ARM64 (M1/M2)

sudo ln -s $(which node) /usr/local/bin/node

Integrating into Your Own App​

The React Native Demo provides a valid configuration that you can use to design your custom app integration.

note

The Movement Library and the SDK Library are two distinct libraries. It is possible to dynamically load models or to update the Model Library independent of the rest of the app.

Install Dependencies​

yarn add react-native-vision-camera@2.x react-native-event-listeners xstate
yarn add @moonlight-labs/rogue-vision-react-native-sdk @moonlight-labs/rogue-vision-movements

Update your application manifest for camera permissions.

Update Configuration to Access SDK Events​

// babel.config.js
module.exports = {
  ...,
  plugins: [
    [
      'react-native-reanimated/plugin',
      {
        globals: ['__scanPoses', '__scanEquipment']
      },
    ],
  ]
}

Import SDK Into App Code​

import { EventRegister } from 'react-native-event-listeners'
import { Movements } from '@moonlight-labs/rogue-vision-movements'
import {
  scanPoses,
  scanEquipment,
  countReps,
} from '@moonlight-labs/rogue-vision-react-native-sdk'

Enumerate the available models​

<ul style={{ columns: 4 }}>
  {Movements.all.map((movement) => (
    <li>
      <strong>{movement.name}</strong>
    </li>
  ))}
</ul>

Start the camera and begin with specific models​

function App() {
  const selectedMovements = Movements.all.map((movement) => {
    return movement
  })

  // Initialize all the selected movements reps to zero
  const [repCount, setCount] = useState(
    selectedMovements.reduce(function (map, movement) {
      map[movement] = 0
      return map
    }, {})
  )

  const repCounter = useFrameProcessor((frame) => {
    'worklet'

    try {
      const inputs = {
        pose: scanPoses(frame),
        equipment: scanEquipment(frame),
      }

      // non-blocking, runs async
      runOnJS(countReps)(inputs, selectedMovements)
    } catch (e) {
      console.log(`Error: ${e.message}`)
    }
  }, [])

  useEffect(() => {
    // this handles the functionality for emitting rep messages

    EventRegister.addEventListener('FrameProcessed', (repData) => {
      // EXPECTED RESULTS
      // [
      //   { type: 'pushups', 'repCount': 1, 'noRepCount': 0, 'queueNoRepCount': 0, event: RepCompleted }
      //   { type: 'squats', 'repCount': 0, 'noRepCount': 1, 'queueNoRepCount': 0, event: NoRep }
      // ]

      repData.map((message, index) => {
        if (message.event == RepEventType.RepCompleted) {
          // increment the rep count for the detected movement
          const newRepCount = message.repCount
          newRepcount[message.type] = repcount
          setCount(newRepCount)

          // update the pose Display
          if (Settings.SHOW_POSE_OVERLAY) updatePose(frameResult.pose)
        } else if (repEvent.event == RepEventType.NoRep) {
          console.log('No rep')
        }
      })
    })
  }, [])

  return (
    <Camera
      frameProcessor={repCounter}
      frameProcessorFps={60}
      {...cameraProps}
    />
  )
}