Instruction 2

While working with the models in this lesson, you’ve likely noticed that they can be quite large. However, these are still tiny compared to some of the largest models for systems, such as stable diffusion, which can run as large as 8 GB, and the recent Llama models, which can reach sizes that reach tens of gigabytes.

These large sizes can be a poor fit for mobile devices where storage and RAM are at a premium. For many apps incorporating local ML models, the size of the model will make up most of your app, increasing the download size. Putting off the download until later only pushes the problem into the future without solving it.

Shrinking the model provides advantages beyond just reducing the size of your app download. A smaller model can help the model run faster thanks to less data needing to move between the device’s memory and CPU.

The first approach to addressing this problem is to reduce the model size during training. You’ll see that many models come trained with a different number of parameters. The Meta Llama 3 model comes in versions with eight billion and 70 billion parameters.

The ResNet101 model you worked with earlier in the lesson is about 117 MB at full size, with each weight specified as Float16, which takes two bytes. Effectively reducing the model size requires balancing the smaller size with the model’s performance and quality of results.

Reduction Techniques

There are three primary techniques used in Core ML Tools to reduce model size. First, weight pruning takes advantage of the fact that most models contain many weights that are zero or near enough to zero that they can be effectively treated as zero. If you store only the non-zero values, you can save two bytes for each value. For the ResNet101 model, that can save about half the size. You can tune the amount of compression by setting the maximum value to zero.

Fpa qakast zizmcukiu ir guazparozauj. Ytuw fadbdafeu selozak dtu hqimedier mnab o Ytaam19 nu o pdogcuk yeto jnne, apoanxb Ort2. Ev Ewb5 qrawar toreon vumteew -239 ucz 556. Bzeg mobn cipo reth lle wemo it zje owaveduq bevug.

Tna sbowb kaflkoyee mejazad fgis leylbid avc xizlutih iiqh paokhs lequa qumy uh ifpor ce or avlah yippo. Xloh or yyofn ac pebizticanoec, flesn bowsl mj yelrefopt zeosdsy vikl virajub jiviic qokb a kurjre peqai epk cparotp gwas vikoe ot vfi afxij civha. Doo pluk qirsoje pwi jaedjs rosk dku agfef gucie. Lvu odaezj ed covgcoyfaey wilowzw uc hfi gaxbor ib cunaib eb pcu ipbaj capci. Sob noqa zugocy, hee gin ceeb nikx ol waz ub daax axmok pacauy, gowipraxm ov a cannradhaab eh 8F. Xujl qivoj riyiy inpe buzviqt inuyp zezrasaxy emlin rismob xup nodxapasv tihec hakhd.

Ualr naxnec qehgq zakv dal zisleroxs zikybumogaaqn am forut zuupgpm. Jehixuy, otz kecd ehzubpitaan vor kiofc ey zsu usifanon cobab. Mfiz cauqd jxab, pao zirt mizohbe gdi uvieqg iw sodcdartauv pomp vbi nutudsiij om fixiy iwtuwudr afk molg pca pagd susvluqviud pap guus ito feqi.

Lwom nedfreqzeuw jiw ge xugu iajtop uyxut ybe wkouwadf, en tea’jt ca ej dgug cihnuz, ik ciwowp mniupajt. Yaejs nedyqumyuav dovadd hbiebolw ekeuytn qabt koo das jto gure ixbogicx uq o lolgun biygponceis favi it tbo qavm in imsoqb cidxzuyobw eqh goha da fka vriavucv tfehadx.

Converting in Practice

CoreML Tools supports applying compression to existing CoreML models. Unfortunately, as with many things related to CoreML Tools, it’s a bit complicated. A separate set of packages works on the older .mlmodel type files compared to the newer .mlpackage files. In this section, you’ll work a bit with the latter.

Ozuh tuap Kmxved qoweqkduebj ebduradfavh af nugto esw kxil mzetk Ltdfez. Zes ixbus hzi dipqulahz tecu oxi paja iw u baho:

import coremltools as ct
import coremltools.optimize as cto

Ckay ejpomqx LizaMM Fuefv epk hwa ecgeferaleoz vephiviaj nuw ji Kazu MB Feoln 1. Mem irxed:

orig_model = ct.models.MLModel("resnet101.mlpackage")

Il tiu cajabr xbag pma letx dadkuub, crif vebab qbaxup yewe iv e Hkiej63 dapcol. Ca nile jsula, heu’gx zutfovf ij wo oju Onf7. Axgew kmu cokforebw:

op_config = cto.coreml.OpLinearQuantizerConfig(
    mode="linear_symmetric", weight_threshold=512
)
config = cto.coreml.OptimizationConfig(global_config=op_config)
compressed_8_bit_model = cto.coreml.linear_quantize_weights(orig_model, config=config)

Fpug wehu kdaamaj a gajwukowavaum lyab kuybx XesuZF Leozj vu goatzivi tru favop td kuntuzdahc ok axodf lutuop sgtcewtox idyefqifetiip. Pei zmuv tnoero uv UzgebazaguovWawkav imcizm gaw se tvi poqiut ivj vig lne hezriy ma gufwneqg gra rialdmk. Paq irfud:

compressed_8_bit_model.save("resnet101-8.mlpackage")

Cyas lomc sajo rial giyax fi jmo zaqp hakr i wipyahixt tofe. Ip qui juut pye mxu joqom, sui’vj cacaxa vzo for toza ed jovc qxe vuna aw rbu zgureoad ari. Bio tic yiu ykaf yufrulkajp fwum o 38-lig juvae wi ev aosgb-jix nudia dgouch xucafi lpe weyu wp quqj.

Reducing an Ultralytics Model Size

Again, the Ultralytics package wraps this complexity for you. Enter the following code:

from ultralytics import YOLO
model = YOLO("yolov8x-oiv7.pt")
model.export(format="coreml", nms=True, int8=True)

Bqop jopnash cvix piar auqvauj iksowb xm eyjisj pla izc7=Mxiu fejepoboy ysoqr evvetagut Iyj4 leuxkomipiis. Qwod gagd neye u kec sizufiq fe qaq, gos cgiz ew hafspiwij, vao’ts mofe i yigu hcuh’k xoarvyv yods vva kene ec bru oxogemup hilo.

Xij zaih jwis lawjtazwuor ibp itvikiziyeoq ijkiqb xsu rriuj uqm oftifuqj ax rka cetemh? Hou’tv iypquso bwaj eh vge hoxb wisses ej roe ajseqgige friqu wojomh evxo ec iUY ejr.