【tensorflow2.0】使用tensorflow-serving部署模型

TensorFlow训练好的模型以tensorflow原生方式保存成protobuf文件后可以用许多方式部署运行。

例如:通过 tensorflow-js 可以用javascrip脚本加载模型并在浏览器中运行模型。

通过 tensorflow-lite 可以在移动和嵌入式设备上加载并运行TensorFlow模型。

通过 tensorflow-serving 可以加载模型后提供网络接口API服务,通过任意编程语言发送网络请求都可以获取模型预测结果。

通过 tensorFlow for Java接口,可以在Java或者spark(scala)中调用tensorflow模型进行预测。

我们主要介绍tensorflow serving部署模型、使用spark(scala)调用tensorflow模型的方法

〇,tensorflow serving模型部署概述

使用 tensorflow serving 部署模型要完成以下步骤。

  • (1) 准备protobuf模型文件。

  • (2) 安装tensorflow serving。

  • (3) 启动tensorflow serving 服务。

  • (4) 向API服务发送请求,获取预测结果。

可通过以下colab链接测试效果《tf_serving》: https://colab.research.google.com/drive/1vS5LAYJTEn-H0GDb1irzIuyRB8E3eWc8

%tensorflow_version 2.x
import tensorflow as tf
print(tf.__version__)
from tensorflow.keras import * 

一,准备protobuf模型文件

我们使用tf.keras 训练一个简单的线性回归模型,并保存成protobuf文件。

import tensorflow as tf
 models,layers,optimizers
 
## 样本数量
n = 800
 
# 生成测试用数据集
X = tf.random.uniform([n,2],minval=-10,maxval=10) 
w0 = tf.constant([[2.0],[-1.0]])
b0 = tf.constant(3.0)
 
Y = X@w0 + b0 + tf.random.normal([n,1],mean = 0.0,stddev= 2.0)  @表示矩阵乘法,增加正态扰动
 
# 建立模型
tf.keras.backend.clear_session()
inputs = layers.Input(shape = (2,),name ="inputs") 设置输入名字为inputs
outputs = layers.Dense(1,name = outputs")(inputs) 设置输出名字为outputs
linear = models.Model(inputs = inputs,outputs = outputs)
linear.summary()
 
# 使用fit方法进行训练
linear.compile(optimizer=rmsprop",loss=msemae"])
linear.fit(X,Y,batch_size = 8,epochs = 100)  
 
tf.print(w = ].kernel)
tf.b = ].bias)
 
# 将模型保存成pb格式文件
export_path = ./data/linear_model/
version = 1"       后续可以通过版本号进行模型版本迭代与管理
linear.save(export_path+version,save_format=tf")
Model: model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
inputs (InputLayer)          [(None,2)]               0         

outputs (Dense)              (None,1)                 3         
=================================================================
Total params: 3
Trainable params: 3
Non-trainable params: 0

Epoch 1/100
100/100 [==============================] - 0s 2ms/step - loss: 273.0472 - mae: 13.9096
Epoch 2/100
100/100 [==============================] - 0s 2ms/step - loss: 250.0846 - mae: 13.3155
Epoch 3/100
100/100 [==============================] - 0s 2ms/step - loss: 228.0106 - mae: 12.7211
Epoch 4/100
100/100 [==============================] - 0s 2ms/step - loss: 208.5060 - mae: 12.1514
Epoch 5/100
100/100 [==============================] - 0s 2ms/step - loss: 188.6825 - mae: 11.5647
Epoch 6/100
100/100 [==============================] - 0s 2ms/step - loss: 170.6377 - mae: 10.9862
Epoch 7/100
100/100 [==============================] - 0s 2ms/step - loss: 153.1913 - mae: 10.4133
Epoch 8/100
100/100 [==============================] - 0s 2ms/step - loss: 137.3440 - mae: 9.8525
Epoch 9/100
100/100 [==============================] - 0s 2ms/step - loss: 122.1956 - mae: 9.2907
Epoch 10/100
100/100 [==============================] - 0s 2ms/step - loss: 108.5923 - mae: 8.7439
Epoch 11/100
100/100 [==============================] - 0s 2ms/step - loss: 94.8144 - mae: 8.1773
Epoch 12/100
100/100 [==============================] - 0s 2ms/step - loss: 83.0037 - mae: 7.6339
Epoch 13/100
100/100 [==============================] - 0s 2ms/step - loss: 71.8595 - mae: 7.1003
Epoch 14/100
100/100 [==============================] - 0s 2ms/step - loss: 61.8016 - mae: 6.5690
Epoch 15/100
100/100 [==============================] - 0s 2ms/step - loss: 52.5519 - mae: 6.0456
Epoch 16/100
100/100 [==============================] - 0s 2ms/step - loss: 44.4070 - mae: 5.5431
Epoch 17/100
100/100 [==============================] - 0s 2ms/step - loss: 37.0890 - mae: 5.0457
Epoch 18/100
100/100 [==============================] - 0s 2ms/step - loss: 30.6758 - mae: 4.5701
Epoch 19/100
100/100 [==============================] - 0s 2ms/step - loss: 25.1626 - mae: 4.1214
Epoch 20/100
100/100 [==============================] - 0s 2ms/step - loss: 20.3433 - mae: 3.6880
Epoch 21/100
100/100 [==============================] - 0s 2ms/step - loss: 16.3199 - mae: 3.2814
Epoch 22/100
100/100 [==============================] - 0s 2ms/step - loss: 13.1249 - mae: 2.9330
Epoch 23/100
100/100 [==============================] - 0s 2ms/step - loss: 10.4714 - mae: 2.6117
Epoch 24/100
100/100 [==============================] - 0s 2ms/step - loss: 8.5397 - mae: 2.3433
Epoch 25/100
100/100 [==============================] - 0s 2ms/step - loss: 7.0484 - mae: 2.1351
Epoch 26/100
100/100 [==============================] - 0s 2ms/step - loss: 6.1226 - mae: 1.9872
Epoch 27/100
100/100 [==============================] - 0s 2ms/step - loss: 5.6070 - mae: 1.9047
Epoch 28/100
100/100 [==============================] - 0s 2ms/step - loss: 5.2954 - mae: 1.8510
Epoch 29/100
100/100 [==============================] - 0s 2ms/step - loss: 5.0835 - mae: 1.8137
Epoch 30/100
100/100 [==============================] - 0s 2ms/step - loss: 4.9148 - mae: 1.7841
Epoch 31/100
100/100 [==============================] - 0s 2ms/step - loss: 4.7715 - mae: 1.7581
Epoch 32/100
100/100 [==============================] - 0s 2ms/step - loss: 4.6395 - mae: 1.7303
Epoch 33/100
100/100 [==============================] - 0s 2ms/step - loss: 4.5205 - mae: 1.7106
Epoch 34/100
100/100 [==============================] - 0s 1ms/step - loss: 4.4232 - mae: 1.6903
Epoch 35/100
100/100 [==============================] - 0s 2ms/step - loss: 4.3417 - mae: 1.6738
Epoch 36/100
100/100 [==============================] - 0s 1ms/step - loss: 4.2691 - mae: 1.6579
Epoch 37/100
100/100 [==============================] - 0s 2ms/step - loss: 4.2078 - mae: 1.6470
Epoch 38/100
100/100 [==============================] - 0s 2ms/step - loss: 4.1606 - mae: 1.6381
Epoch 39/100
100/100 [==============================] - 0s 2ms/step - loss: 4.1203 - mae: 1.6292
Epoch 40/100
100/100 [==============================] - 0s 2ms/step - loss: 4.0847 - mae: 1.6230
Epoch 41/100
100/100 [==============================] - 0s 2ms/step - loss: 4.0589 - mae: 1.6182
Epoch 42/100
100/100 [==============================] - 0s 2ms/step - loss: 4.0382 - mae: 1.6141
Epoch 43/100
100/100 [==============================] - 0s 2ms/step - loss: 4.0188 - mae: 1.6109
Epoch 44/100
100/100 [==============================] - 0s 2ms/step - loss: 4.0089 - mae: 1.6098
Epoch 45/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9979 - mae: 1.6075
Epoch 46/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9891 - mae: 1.6055
Epoch 47/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9848 - mae: 1.6053
Epoch 48/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9806 - mae: 1.6044
Epoch 49/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9752 - mae: 1.6037
Epoch 50/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9739 - mae: 1.6038
Epoch 51/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9712 - mae: 1.6024
Epoch 52/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9690 - mae: 1.6024
Epoch 53/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9685 - mae: 1.6021
Epoch 54/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9667 - mae: 1.6021
Epoch 55/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9651 - mae: 1.6009
Epoch 56/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9656 - mae: 1.6019
Epoch 57/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9635 - mae: 1.6016
Epoch 58/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9640 - mae: 1.6012
Epoch 59/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9655 - mae: 1.6018
Epoch 60/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9639 - mae: 1.6016
Epoch 61/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9650 - mae: 1.6010
Epoch 62/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9651 - mae: 1.6017
Epoch 63/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9646 - mae: 1.6021
Epoch 64/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9638 - mae: 1.6019
Epoch 65/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9639 - mae: 1.6027
Epoch 66/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9656 - mae: 1.6013
Epoch 67/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9645 - mae: 1.6019
Epoch 68/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9635 - mae: 1.6024
Epoch 69/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9637 - mae: 1.6015
Epoch 70/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9643 - mae: 1.6022
Epoch 71/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9626 - mae: 1.6022
Epoch 72/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9632 - mae: 1.6015
Epoch 73/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9638 - mae: 1.6023
Epoch 74/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9643 - mae: 1.6017
Epoch 75/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9638 - mae: 1.6003
Epoch 76/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9648 - mae: 1.6022
Epoch 77/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9624 - mae: 1.6023
Epoch 78/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9637 - mae: 1.6019
Epoch 79/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9644 - mae: 1.6019
Epoch 80/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9648 - mae: 1.6018
Epoch 81/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9649 - mae: 1.6025
Epoch 82/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9631 - mae: 1.6021
Epoch 83/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9650 - mae: 1.6020
Epoch 84/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9640 - mae: 1.6020
Epoch 85/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9644 - mae: 1.6014
Epoch 86/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9638 - mae: 1.6017
Epoch 87/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9638 - mae: 1.6024
Epoch 88/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9646 - mae: 1.6016
Epoch 89/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9643 - mae: 1.6016
Epoch 90/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9636 - mae: 1.6019
Epoch 91/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9637 - mae: 1.6029
Epoch 92/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9644 - mae: 1.6026
Epoch 93/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9637 - mae: 1.6014
Epoch 94/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9623 - mae: 1.6019
Epoch 95/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9637 - mae: 1.6015
Epoch 96/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9641 - mae: 1.6017
Epoch 97/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9635 - mae: 1.6027
Epoch 98/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9644 - mae: 1.6024
Epoch 99/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9640 - mae: 1.6021
Epoch 100/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9638 - mae: 1.6024
w =  [[1.99997306]
 [-1.01220131]]
b =  [2.88236618]
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/resource_variable_ops.py:1817: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
INFO:tensorflow:Assets written to: ./data/linear_model/1/assets
 查看保存的模型文件
!ls {export_path+version}

assets saved_model.pb variables

 查看模型文件相关信息
!saved_model_cli show --dir {export_path+str(version)} --all
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:

signature_def[__saved_model_init_op]:
  The given SavedModel SignatureDef contains the following input(s):
  The given SavedModel SignatureDef contains the following output(s):
    outputs[] tensor_info:
        dtype: DT_INVALID
        shape: unknown_rank
        name: NoOp
  Method name is: 

signature_def[serving_default]:
  The given SavedModel SignatureDef contains the following input(s):
    inputs[] tensor_info:
        dtype: DT_FLOAT
        shape: (-1,2)
        name: serving_default_inputs:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs[)
        name: StatefulPartitionedCall:0
  Method name is: tensorflow/serving/predict
WARNING: Logging before flag parsing goes to stderr.
W0413 05:10:30.262132 140384690243456 deprecation.py:506] From /usr/local/lib/python2.7/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1786: calling _constraint arguments to layers.

Defined Functions:
  Function Name: __call__
    Option 1
      Callable with:
        Argument 1
          inputs: TensorSpec(shape=(None,2),dtype=tf.float32,name=u)
        Argument 2
          DType: bool
          Value: False
        Argument 3
          DType: NoneType
          Value: None
    Option           DType: bool
          Value: True
        Argument           DType: NoneType
          Value: None

  Function Name: _default_save_signature)

  Function Name: call_and_return_all_conditional_losses          DType: NoneType
          Value: None

二,安装 tensorflow serving

安装 tensorflow serving 有2种主要方法:通过Docker镜像安装,通过apt安装。

通过Docker镜像安装是最简单,最直接的方法,推荐采用。

Docker可以理解成一种容器,其上面可以给各种不同的程序提供独立的运行环境。

一般业务中用到tensorflow的企业都会有运维同学通过Docker 搭建 tensorflow serving.

无需算法工程师同学动手安装,以下安装过程仅供参考。

不同操作系统机器上安装Docker的方法可以参照以下链接。

Windows: https://www.runoob.com/docker/windows-docker-install.html

MacOs: https://www.runoob.com/docker/macos-docker-install.html

CentOS: https://www.runoob.com/docker/centos-docker-install.html

安装Docker成功后,使用如下命令加载 tensorflow/serving 镜像到Docker中

docker pull tensorflow/serving

三,启动 tensorflow serving 服务

!docker run -t --rm -p 8501:8501 \
    -v /Users/.../data/linear_model/ \
    -e MODEL_NAME=linear_model \
    tensorflow/serving & >server.log 2>&1

四,向API服务发送请求

可以使用任何编程语言的http功能发送请求,下面示范linux的 curl 命令发送请求,以及Python的requests库发送请求。

!curl -d {"instances": [1.0,2.0,5.0]} \
    -X POST http://localhost:8501/v1/models/linear_model:predict
{
    predictions": [[3.06546211],[5.01313448]
    ]
}
 json,requests
 
data = json.dumps({signature_name": instances": [[1.0,2.0],[5.0,7.0]]})
headers = {content-typeapplication/json}
json_response = requests.post(http://localhost:8501/v1/models/linear_model:predict,data=data,headers=headers)
predictions = json.loads(json_response.text)[]
print(predictions)

 

参考:

开源电子书地址:https://lyhue1991.github.io/eat_tensorflow2_in_30_days/

GitHub 项目地址:https://github.com/lyhue1991/eat_tensorflow2_in_30_days

版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。

相关推荐


MNIST数据集可以说是深度学习的入门,但是使用模型预测单张MNIST图片得到数字识别结果的文章不多,所以本人查找资料,把代码写下,希望可以帮到大家~1#BudingyourfirstimageclassificationmodelwithMNISTdataset2importtensorflowastf3importnumpyasnp4impor
1、新建tensorflow环境(1)打开anacondaprompt,输入命令行condacreate-ntensorflowpython=3.6注意:尽量不要更起名字,不然环境容易出错在选择是否安装时输入“y”(即为“yes”)。其中tensorflow为新建的虚拟环境名称,可以按喜好自由选择。python=3.6为指定python版本为3
这篇文章主要介绍“张量tensor是什么”,在日常操作中,相信很多人在张量tensor是什么问题上存在疑惑,小编查阅了各式资料,整理出简单好用的操作方法,希望对大...
tensorflow中model.fit()用法model.fit()方法用于执行训练过程model.fit(训练集的输入特征,训练集的标签,batch_size,#每一个batch的大小epochs,#迭代次数validation_data=(测试集的输入特征,
https://blog.csdn.net/To_be_little/article/details/124438800 目录1、查看GPU的数量2、设置GPU加速3、单GPU模拟多GPU环境1、查看GPU的数量importtensorflowastf#查看gpu和cpu的数量gpus=tf.config.experimental.list_physical_devices(device_type='GPU')cpus=tf.c
根据身高推测体重const$=require('jquery');consttf=require('@tensorflowfjs');consttfvis=require('@tensorflowfjs-vis');/*根据身高推测体重*///把数据处理成符合模型要求的格式functiongetData(){//学习数据constheights=[150,151,160,161,16
#!/usr/bin/envpython2#-*-coding:utf-8-*-"""CreatedonThuSep610:16:372018@author:myhaspl@email:myhaspl@myhaspl.com二分法求解一元多次方程"""importtensorflowastfdeff(x):y=pow(x,3)*3+pow(x,2)*2-19return
 继续上篇的pyspark集成后,我们再来看看当今热的不得了的tensorflow是如何继承进pycharm环境的参考:http://blog.csdn.net/include1224/article/details/53452824思路其实很简单,说下要点吧1.python必须要3.564位版本(上一篇直接装的是64位版本的Anaconda)2.激活3.5版本的
首先要下载python3.6:https://www.python.org/downloadselease/python-361/接着下载:numpy-1.13.0-cp36-none-win_amd64.whl 安装这两个:安装python3.6成功,接着安装numpy.接着安装tensorflow: 最后测试一下: python3.6+tensorflow安装完毕,高深的AI就等着你去
参考书《TensorFlow:实战Google深度学习框架》(第2版)以下TensorFlow程序完成了从图像片段截取,到图像大小调整再到图像翻转及色彩调整的整个图像预处理过程。#!/usr/bin/envpython#-*-coding:UTF-8-*-#coding=utf-8"""@author:LiTian@contact:694317828@qq.com
参考:TensorFlow在windows上安装与简单示例写在开头:刚开始安装的时候,由于自己的Python版本是3.7,安装了好几次都失败了,后来发现原来是tensorflow不支持3.7版本的python,所以后来换成了Python3.6,就成功了。。。。。anconda:5.3.2python版本:3.6.8tensorflow版本:1.12.0安装Anconda
实验介绍数据采用CriteoDisplayAds。这个数据一共11G,有13个integerfeatures,26个categoricalfeatures。Spark由于数据比较大,且只在一个txt文件,处理前用split-l400000train.txt对数据进行切分。连续型数据利用log进行变换,因为从实时训练的角度上来判断,一般的标准化方式,
 1)登录需要一个 invitationcode,申请完等邮件吧,大概要3-5个小时;2)界面3)配置数据集,在右边列设置 
模型文件的保存tensorflow将模型保持到本地会生成4个文件:meta文件:保存了网络的图结构,包含变量、op、集合等信息ckpt文件:二进制文件,保存了网络中所有权重、偏置等变量数值,分为两个文件,一个是.data-00000-of-00001文件,一个是.index文件checkpoint文件:文本文件,记录了最新保持
原文地址:https://blog.csdn.net/jesmine_gu/article/details/81093686这里只是做个收藏,防止原链接失效importosimportnumpyasnpfromPILimportImageimporttensorflowastfimportmatplotlib.pyplotaspltangry=[]label_angry=[]disgusted=[]label_d
 首先声明参考博客:https://blog.csdn.net/beyond_xnsx/article/details/79771690?tdsourcetag=s_pcqq_aiomsg实践过程主线参考这篇博客,相应地方进行了变通。接下来记载我的实践过程。  一、GPU版的TensorFlow的安装准备工作:笔者电脑是Windows10企业版操作系统,在这之前已
1.tensorflow安装  进入AnacondaPrompt(windows10下按windows键可找到)a.切换到创建好的tensorflow36环境下:activatetensorflow36    b.安装tensorflow:pipinstlltensorflow    c.测试环境是否安装好       看到已经打印出了"h
必须走如下步骤:sess=tf.Session()sess.run(result)sess.close()才能执行运算。Withtf.Session()assess:Sess.run()通过会话计算结果:withsess.as_default():print(result.eval())表示输出result的值生成一个权重矩阵:tf.Variable(tf.random_normal([2,3]
tf.zeros函数tf.zeros(shape,dtype=tf.float32,name=None)定义在:tensorflow/python/ops/array_ops.py.创建一个所有元素都设置为零的张量. 该操作返回一个带有形状shape的类型为dtype张量,并且所有元素都设为零.例如:tf.zeros([3,4],tf.int32)#[[0,0,
一、Tensorflow基本概念1、使用图(graphs)来表示计算任务,用于搭建神经网络的计算过程,但其只搭建网络,不计算2、在被称之为会话(Session)的上下文(context)中执行图3、使用张量(tensor)表示数据,用“阶”表示张量的维度。关于这一点需要展开一下       0阶张量称