Virtual Try-On Custom HTML Element

Introduction

The Virtual Try On (VTO) experience is designed for high-end mobile devices, allowing users to see virtual shoes on their feet.

Quick Start

Import the component into your HTML:

<script
  type="module"
  src="https://sneaker-window.vyking.io/vyking-apparel/1/vyking-apparel.js"
></script>

Use the Vyking Apparel component by adding the following HTML:

<vyking-apparel
  share
  autocamera
  autocamera-width="640"
  autocamera-height="360"
  onerror="alert(`Error: ${event.message}`)"
  apparel="https://sneaker-window.vyking.io/vyking-assets/customer/vyking-io/yeezy_boost_700_carbon_blue/offsets.json"
  config="../assets/config/modeld.foot.bin"
  config-key="io.vyking"
  alt="Virtual Try On"
>
  <canvas slot="canvas">Virtual Try On</canvas>
</vyking-apparel>

Examples

Configuration File

A configuration file, supplied by Vyking, is required. This file will be encrypted with a client key and will only be valid for certain domain names. The choice of client key and supported domains will be provided by you, the client. The page using the vyking-apparel library must supply this client key along with the fetched contents of the configuration file and be hosted on one of the valid domain names. Only one key is supported, but several domain names are allowed. To support private development servers, private IP addresses (e.g. 192.168.0.20) and localhost are automatically allowed, but the client key must be valid.

The configuration file will have an expiry date, and needs to be replaced before this date is reached to prevent interruption to the service.

Disposal

Because vyking-apparel uses Three and Tensorflow it has resources that need to be explicitly disposed of when finished with. Just removing the vyking-apparel element from the document is not enough, the vyking-apparel dispose() method needs to be called. However, It is possible to maintain reference to a vyking-apparel element and just keep adding it and removing it from the document as required.

Memory and CPU Usage

Due to the nature of AR and 3D graphics both the VTO and 3D viewer can have high demands on both memory and CPU. This needs to be considered when deploying on mobile devices as they have limited memory and cpu resources. The use of iframes helps encapsulate the viewers and ensures greater release of memory when the iframe is closed. If memory or CPU usage is still an issue then popup windows, which appear as tabs on mobile devices, can be considered as when configured correctly they run as separate processes on mobile devices.

Tensorflow Delegates and Model Compilation Time

Camera image frames are processed using Tensorflow models and both the WebGL and CPU backends are supported. The choice is determined by the HTMLVykingApparelElement.imageProcessorDelegate global configuration option, as shown in the varius examples. Although the WebGL backend tends to be faster it can take a long time to compile the model's shaders the very first time they are loaded after a device restart. To alleviate this delay a preWarmTensorflowModel.js script is provided that can be launched early to load and compile the Tensorflow model before the user launches a VTO experience. The various PDP examples provided demonstrate how the script can be loaded and run and only present the VTO button once the script has completed.

Camera management

When the autocamera attribute is set the window onvisibilitychange event is used to manage the camera. This is so that the camera is only in use whilst visible and is free for use by other browser tabs when not. The events are only listened for when the vyking-apparel element is connected to the DOM to prevent automatically starting the camera when the element is not visible.

Viewport meta tag

Consider adding a viewport meta tag to the HTML page hosting the vyking-apparel element. (This maybe easier if the vyking-apparel element is hosted in an iframe.) Typically this would be:

<head>
  :
  :
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
  :
  :
</head>

Releases

1.1.6 First release
1.1.7 Added lensFactor attribute and some documentation
1.1.8 Support HDR environments, Upgraded to Three 0.152.0 & started using Lit (version 2.7.5)
1.1.9 Refactored events so they bubble and are composable. Also changed play, pause and finish from CustomEvents to Events.
1.1.10 Support 42 point Tensorflow model.
1.2.1 Default loader, advice, play button can be replaced. Default share button added.
1.2.2 Support offsets.json version 2 files.
1.2.3 Now using @vyking/apparel version 1.1.12.
1.2.4 Now using @vyking/apparel version 1.1.13. Renamed attribute autocamera-framerate to autocamera-frame-rate and now support attribute autocamera-facing-mode.
1.2.5 Support for GLTF variants.
1.2.6 Upgraded to Tensorflow 4.11.0 and Three 0.157.0
1.2.7 Upgraded to @vyking/apparel version 1.1.16.
**`1.2.8 Upgraded to @vyking/apparel version 1.1.17.
**`1.2.9 Upgraded to @vyking/apparel version 1.1.18 and set the defaultEnvironmentImage property to reference a neutral.hdr image so that legacy offsets.json files are not dark.
**`1.2.10 Now also support the attribute "config-key" for setting the decryption key. Upgraded to @vyking/apparel version 1.1.19.
**`1.2.11 Upgraded to @vyking/apparel version 1.1.20.
**`1.2.12 Upgraded to @vyking/apparel version 1.1.21 and also stop/start the camera on th blur/focus events for iOS to try and handle iOS multi-lens camera bugs.
1.2.13
- Upgraded to @vyking/apparel version 1.1.22.
- Changed the default tone mapping to no tone mapping.
- Added support for share decorations.
- The blur and focus events used to control the autocamera are now only active if the vyking-apparel element is connected to the DOM.The image is assigned to the vyking-apparel's "backgroundImage" style property.
1.2.14
- Fixed the blur and focus events controlling the autocamera to work when vyking-apparel is hosted inside an iframe.
- Changed the default advice image.
1.2.15
- Upgraded to @vyking/apparel version 1.1.23.
- The qrCode's img element can now be styled.
1.2.16
- Upgraded to @vyking/apparel version 1.1.24.
- Reverted the default tone mapping back to ACESFilmicToneMapping.
1.2.17
- Upgraded to @vyking/apparel version 1.1.25 to get the new improved foot tracking.
1.2.18
- Upgraded to @vyking/apparel version 1.1.26 to improve Android performance and fix some memory leaks.
1.2.19
- Stopped the share sheet from pausing the VTO experience on Android devices.
1.2.20
- Added a runShare() method
- Replaced the blur & focus event handlers with a visibilitychange handler.
1.2.21
- Added disabledQRCodeUrl and disabledQRCodeCaption attributes.
1.2.22
- Hidden and disabled play/pause controls is now the default.
1.2.23
- Fixed an issue where the DracoLoader was not being disposed of correctly.
1.2.24
- Upgraded to @vyking/apparel version v1.1.28
- Also dispose of the dracoLoader when vyking-apparel is removed from the document to free up memory.
- Changed the sampling rate to be controlled by the auto camera-framerate.
1.2.25
- Upgraded to @vyking/apparel version 1.1.29.
- Only create the DracoLoader once, because it leaks memory if disposed of and then recreated on demand.
- Change the console.info() calls to only report strings.
1.2.26
- Upgraded to @vyking/apparel version 1.1.30
1.3.1
- Upgraded to @vyking/apparel version 1.1.31
- QRCode caption can now contain HTML.
- Switched to a default loader showing progress.
- Changed the share feature to show a preview of the image that will be shared.
1.3.2
- Now using the default cache request header when fetching the modeld.bin file because the 'no-cache' option now causes iOS Safari to fetch the file every time.
1.3.3
- Upgraded to @vyking/apparel version 1.1.32 allowing signed urls to access offsets.json file and their resources.
1.3.4
- Various accessibility improvements.
1.3.5
- Upgraded to @vyking/apparel version 1.1.33.
- The QRCode image alt text is now customizable.
- There is now a preWarmTensorflowModel.js script that can be used to fetch and warm a tensorflow model before the VTO is launched.
1.3.6
- Upgraded to @vyking/apparel version 1.1.34.
- Do tensorflow data() calls in the preWarmTensorflowModel.js script because this also does some warmup work.
1.3.7
- Upgraded to @vyking/apparel version 1.1.35.
- Upgraded to Tensorflow version 4.22.0
- Upgraded to Three js version 0.165.0
- Added support for Tensorflow's WASM backend, and set as the default.

Attributes

Name	Description	Type	Default
config	A configuration buffer must be provided to unlock the tracking features. This can either be specified as a url via this attribute or by setting the configPromise property with a promise that will resolve to the configuration as an arrayBuffer type. The latter option allows for prefetching the configuration.	.bin File URL
config-key	A key needs to be provided to decrypt the configuration buffer.	String
~~key~~	This is deprecated, use config-key instead. A key needs to be provided to decrypt the configuration buffer.	String
autocamera	Automatically start and manage the camera.	Boolean	`false`
autocamera-width	The ideal width to use when automatically managing the camera.	Number	`640`
autocamera-height	The ideal height to use when automatically managing the camera.	Number	`360`
autocamera-frame-rate	The max frame rate to use when automatically managing the camera.	Number	`60`
autocamera-facing-mode	The facing mode to use when automatically managing the camera.	`environment` or `user`	`environment`
poster	An image to display whilst waiting for the resources to load and start.	Image URL
apparel	The virtual apparel to try on.	JSON File URL
flipy	If specified the image will be mirrored by flipping about the Y axis.	Boolean	`false`
rotate	Specify this when the input camera needs rotating. This is typically when an external camera needs to be rotated from landscape to portrait.	Boolean	`false`
stats	Enable the Three.js statistics element and console logging of internal timings.	Boolean	`false`
debug	Enable debug features such as showing the various canvases used.	Boolean	`false`
lens-factor	Adjust the zoom factor used when rendering the image. Zooming in slightly allows the virtual apparel to still be seen when the tracked body part appears to nearly outside the camera's field of view. Values less that 1.0 zoom in and values greater than 1.0 zoom out.	Float	`1.0`
share	Enable the share feature. This uses the navigator.share facility, if supported.	Boolean	`false`
share-quality	The share feature generates an 'image/jpeg' image of the rendered canvas using the Canvas.toDataURL method. The share-quality attribute is used as the method's quality parameter.	Float, `0.0` to `1.0` (inclusive)	`1.0`
share-decoration	If this attribute is present then the share decorations will be added to the shared image.	Boolean	`false`
share-product	The share product name decoration to use instead of the longDescription property in the offsets.json file.
share-brand	The share brand name decoration to use instead of the shortDescription property in the offsets.json file.
share-logo	A document.querySelector tag referencing an img element defining the share logo decoration to use.
default-tone-mapping	The tone mapping to use if not specified in the offsets.json file.	String	`NoToneMapping`
default-exposure	The tone mapping exposure value to use if not specified in the offsets.json file.	Number	`1.0`
default-environment-image	The url of an equirectangular HDR environment image to use if not specified in the offsets.json file, or if the offset.json file only specifies a jpg image (default: https://sneaker-window.vyking.io/vyking-apparel/1/assets/images/neutral.hdr).	String	`https://sneaker-window.vyking.io/vyking-apparel/1/assets/images/neutral.hdr`
class	Will contain "busy" whilst loading resources such as shoe models and "detected" whilst body parts are detected.
disabledQRCodeUrl	This overrides the self.HTMLVykingApparelElement.disabledQRCodeUrl URL used for generating the QR code when vyking-apparel is disabled. The QR code is generated when vyking-apparel is attached to the document.
disabledQRCodeCaption	This overrides the self.HTMLVykingApparelElement.disabledQRCodeCaption text used for the QR code caption when vyking-apparel is disabled. The QR code is generated when vyking-apparel is attached to the document.
disabledQRCodeAltText	This overrides the self.HTMLVykingApparelElement.disabledQRCodeAltText text used for the QR code image 'alt' attribute.

Properties

Name	Description	Type
configPromise	Set the configuration as a promise that will resolve to an ArrayBuffer type. This allows a pre-fetched configuration to be used.	(config: Promise)
config	Set or unset the config property.	(url: `string` or `null`)
configKey	Set or unset the key property.	(key: `string` or `null`)
~~key~~	This is deprecated, use configKey instead. Set or unset the key property.	(key: `string` or `null`)
apparel	Set or unset the apparel property.	(url: `string` or `null`)
srcVideo	Get the video element providing the images to process.	`HTMLVideoElement` or `undefined`
canvas	Get the canvas element being rendered to.	`HTMLCanvasElement` or `undefined`
expiryDate	Get the configuration expiry date.	`Date`
paused	Returns a boolean that indicates whether the vyking-apparel element is paused.	`boolean`

Parts

Name	Description
default-advice	Scope your CSS to vyking-apparel::part(advice) to change the styling of the advice. Most common would be probably changing the background-color, height and width.
default-progress-container	Scope your CSS to vyking-apparel::part(default-progress-container) to change the styling of the default loader container div. The individual components of the default progress loader can be styled by targeting the parts: default-progress-img, default-progress-bar, default-progress-percentage and default-progress-test.
default-progress-img	Scope your CSS to vyking-apparel::part(default-progress-img) to change the styling of the default loader's image.
default-progress-bar	Scope your CSS to vyking-apparel::part(default-progress-bar) to change the styling of the default loader's indicator bar.
default-progress-percentage	Scope your CSS to vyking-apparel::part(default-progress-percentage) to change the styling of the default loader's percentage label.
default-progress-text	Scope your CSS to vyking-apparel::part(default-progress-text) to change the styling of the default loader's text.
default-share	Scope your CSS to vyking-apparel::part(default-share) to change the styling of the default share button.
default-share-image-wrapper	The share image preview container div can be styled using the vyking-apparel::part(default-share-image-wrapper) CSS selector. The preview components can be styled by targeting the default-share-image-img, default-share-image-close and the default-share-image-share selectors.
default-share-image-img	Scope your CSS to vyking-apparel::part(default-share-image-img) to change the styling of the default share preview image.
default-share-image-close	Scope your CSS to vyking-apparel::part(default-share-image-close) to change the styling of the default share preview close button.
default-share-image-share	Scope your CSS to vyking-apparel::part(default-share-image-share) to change the styling of the default share preview share button.
default-play	Scope your CSS to vyking-apparel::part(play) to change the styling of the play button. By default the play and pause buttons are hidden, but can be enabled by defining both the vyking-apparel::part(play) and the vyking-apparel::part(pause) CSS rules with the attribute 'display: inline-block;'.
default-pause	The pause feature, disabled by default, can be enabled by setting 'display: inline-block' on the vyking-apparel::part(pause) CSS rule.
qrcode	Scope your CSS to vyking-apparel::part(qrcode) to change the styling of the qrcode. Most common would be probably changing the height and width.
qrcode-img	The QR code's img element can be styled by scoping your CSS to vyking-apparel::part(qrcode-img).
qrcodeCaption	The QR code's figcaption element can be styled by scoping your CSS to vyking-apparel::part(qrcodeCaption).

Slots

Name	Description
loader	By placing a child element under <vyking-apparel> with slot="loader", this element will replace the default loader. This is expected to be a CSS style only loader that ideally only uses compositing operations.
advice	By placing a child element under <vyking-apparel> with slot="advice", this element will replace the default advice displayed when tracked body parts are not detected.
play	By placing a child element under <vyking-apparel> with slot="play", this element will replace the default play button displayed when the experience is paused.
share	By placing a child element under <vyking-apparel> with slot="share", this element will replace the default share button.
src	By placing a child element under <vyking-apparel> with slot="src", this element will replace the default HTMLVideoElement. An HTMLVideoElement provides the source images to place the virtual apparel on. This can either specify a normal video, a custom srcObject or if the autocamera attribute is specified then a camera will be automatically attached to the video element and managed. If a custom srcObject is supplied then the vyking-apparel element will not manage the video element content (eg will not play or pause the video element). Set the hidden attribute to prevent processing delays from affecting the layout. Set the autoplay, muted, and playsinline attributes to allow the video to play.
canvas	The canvas the VTO experience will be rendered on. This must be provided, there is no default.

Methods

Name	Description
dispose	Dispose of the internal resources. Just detaching a vyking-apparel instance from the document is not enough to allow all internal resources to be garbage collected.
pause	Pause the VTO experience.
play	Play the VTO experience.
runShare(showImageFirst: boolean = false)	Share an image capture of the current VTO experience. Returns a Promise. If showImageFirst is true a preview of the image that will be shared is show first.

Events

Name	Description
error	An error has occurred. An interesting example is when errorEvent.error.cause.name === 'NotAllowedError', because this means the user denied camera permission.
progress	Loading progress event. `detail: { percentage: percentage }`
expirydate	The date the configuration will expire.
anatomydetected	This event is dispatched when detection transitions between detecting and not detecting, or visa-versa, a body part. `detail: { bodyPart: string, detected: boolean }`
apparelchanged	Dispatched when the request to change the apparel has completed. `detail: { success: boolean, cause?: Error }`
apparelmaterialaltered	Dispatched when the request to alter the apparel material has completed. `detail: { success: boolean, cause?: Error }`
imageprocessorchanged	Dispatched when the image processor has been loaded. `detail: { success: boolean, cause?: Error }`
play	Dispatched when transitioning to playing.
pause	Dispatched when the vyking-apparel element is paused.
finish	Dispatched when finished, usually when dispose() is called.

Global Configuration

Some configurable properties of the vyking-apparel element must be defined before the first vyking-apparel element is created, so set the configuration early. A global configuration option to do this can be define like so:

<script>
  self.HTMLVykingApparelElement = self.HTMLVykingApparelElement || {};
</script>

powerPreference

This controls the Three.js WebGLRender's power preference setting.

<script>
  self.HTMLVykingApparelElement = self.HTMLVykingApparelElement || {};
  self.HTMLVykingApparelElement.powerPreference = "high-performance";
</script>

imageProcessorDelegate

This controls which Tensorflow backend is used to process the camera images. The options are: "CPU" or "GPU". The default, if not specified, is CPU.

    <script>
        self.HTMLVykingApparelElement = self.HTMLVykingApparelElement || {}
        self.HTMLVykingApparelElement.imageProcessorDelegate = 'CPU';
    </script>

isDisabled, disabledQRCodeUrl, disabledQRCodeAltText & disabledQRCodeCaption

The isDisabled option can be combined with the disabledQRCodeUrl and disabledQRCodeCaption, disabledQRCodeAltText options to display a QR code instead of the VTO experience under controlled conditions. For example, instead of showing the virtual shoe fitting experience on a desktop computer a QR code can be displayed that the user can point their mobile device at to run on instead.

<script>
  const isDisabled = () => {
    if (
      /android/i.test(navigator.userAgent) ||
      /iPad|iPhone|iPod/.test(navigator.userAgent)
    ) {
      return false;
    }

    return false;
  };
  self.HTMLVykingApparelElement = self.HTMLVykingApparelElement || {};
  self.HTMLVykingApparelElement.disabledQRCodeUrl = document.location.href;
  self.HTMLVykingApparelElement.disabledQRCodeCaption = '<div style="color:#878787;margin:0;padding:0;font-family: sans-serif;font-weight: 500;font-size: 12px;line-height: 100%;letter-spacing: 0%;text-align: center;vertical-align: middle;">Try on</div><hr style="height:1px;background-color:#CCCCCC;border-width:0;width:80%;"><p style="font-family: sans-serif;font-weight: 500;font-size: 18px;line-height: 100%;letter-spacing: 0%;text-align: center;vertical-align: middle;">Scan with your smartphone</p>'
  self.HTMLVykingApparelElement.disabledQRCodeAltText = 'Custom alt text for the QR code'
  self.HTMLVykingApparelElement.imageProcessorDelegate = 'CPU'
  self.HTMLVykingApparelElement.isDisabled = isDisabled();
</script>

The QR code is created and shown when the vyking-apparel element is added to the document. Setting or changing the attributes disabledQRCodeUrl and disabledQRCodeCaption allows the QR code url and caption to be changed per vyking-apparel instance, but the effect will only be visible when the element is attached to the document.

Introduction​

Quick Start​

Examples​

Configuration File​

Disposal​

Memory and CPU Usage​

Tensorflow Delegates and Model Compilation Time​

Camera management​

Viewport meta tag​

Releases​

Attributes​

Properties​

Parts​

Slots​

Methods​

Events​

Global Configuration​

powerPreference​

imageProcessorDelegate​

isDisabled, disabledQRCodeUrl, disabledQRCodeAltText & disabledQRCodeCaption​