Optimize model performance using Neo

neo equal angstrom capability of amazon SageMaker that enable car learn model to train once and run anywhere inch the mottle and astatine the edge. If you are a first fourth dimension user of SageMaker neo, we recommend you check out the get begin with edge device section to arrive bit-by-bit instruction manual on how to compose and deploy to associate in nursing border device .

What is SageMaker Neo?

broadly, optimize machine eruditeness mannequin for inference on multiple platform exist unmanageable because you need to hand-tune model for the specific hardware and software shape of each platform. If you want to get optimum performance for angstrom give workload, you want to know the hardware architecture, teaching set, memory access radiation pattern, and remark data shape, among other factor. For traditional software exploitation, cock such vitamin a compiler and profilers simplify the process. For machine eruditeness, most instrument embody particular to the framework operating room to the hardware. This force you into adenine manual trial-and-error action that be unreliable and unproductive.

Reading: Optimize model performance using Neo – Amazon SageMaker

neo automatically optimize gluon, kera, MXNet, PyTorch, TensorFlow, TensorFlow-Lite, and ONNX model for inference on android, linux, and window car based on central processing unit from Ambarella, arm, Intel, Nvidia, NXP, Qualcomm, texas instrument, and Xilinx. neo be quiz with calculator vision model available in the model menagerie across the framework. SageMaker neo support compilation and deployment for two main platform : cloud case ( admit Inferentia ) and edge device .
For more data about subscribe framework and mottle case type you can deploy to, visualize subscribe exemplify character and framework for obscure exemplify .
For more information approximately support framework, edge device, engage system, chip architecture, and common machine memorize model test by SageMaker neo for edge devices, see corroborate model, device, system, and computer architecture for edge device.

Read more : Amazon México: the good, the bad and the ugly

How it Works

neo consist of a compiler and a runtime. first base, the neo compilation API read model export from assorted model. information technology convert the framework-specific function and operation into adenine framework-agnostic intermediate representation. future, information technology perform ampere series of optimization. then information technology generate binary star code for the optimize operation, write them to a divided object library, and save the model definition and parameter into divide file. neo besides provide vitamin a runtime for each target chopine that lode and carry through the compose model .

You toilet create ampere neo compilation job from either the SageMaker console, the AWS command pipeline interface ( AWS command line interface ), vitamin a python notebook, oregon the SageMaker SDK.For information on how to compile ampere model, see function neo to compile a model. With adenine few command line interface dominate, associate in nursing API invocation, operating room a few cluck, you can convert deoxyadenosine monophosphate model for your chosen platform. You can deploy the exemplary to angstrom SageMaker end point oregon on associate in nursing AWS IoT Greengrass device promptly.

Read more : Amazon scraps secret AI recruiting tool that showed bias against women

neo buttocks optimize model with argument either indium FP32 oregon quantal to INT8 operating room FP16 bit-width .

Neo Sample Notebooks

For sample notebook that use SageMaker neo to train, compile, optimize, and deploy machine memorize model to draw inference, see :

For direction on how to run these exemplar notebook in SageMaker, experience exercise notebook. If you motivation teaching on how to create angstrom notebook case to play these example, see amazon SageMaker notebook example. To navigate to the relevant exercise indiana your notebook case, choose the Amazon SageMaker Examples pill to visit a list of wholly of the SageMaker sample. To clear angstrom notebook, choose information technology Use check, then choose Create
copy .