656 questions
0
votes
0
answers
68
views
Error using tf.estimator.LinearClassifier() in tensorflow 2.16
I can't use tf.estimator in my code with tensorflow 2.16.
Below is the error:
AttributeError Traceback (most recent call last)
Cell In[17], line 1
----> 1 linear_est = tf....
0
votes
3
answers
6k
views
from tensorflow.compat.v1 import estimator as tf_estimator ImportError: cannot import name 'estimator' from 'tensorflow.compat.v1'
I'm doing an object detection project on the raspi-5 and i run this code on the cmd line
python Tensorflow/models/research/object_detection/model_main_tf2.py --model_dir=Tensorflow/workspace/models/...
-1
votes
1
answer
104
views
Second call to tf.estimator.train_and_evaluate finished after 1 step of training
I am running two tensorflow models using the tf.estimator.train_and_evaluate function one after the other.
# The first
train_spec = tf.estimator.TrainSpec(input_fn=my_input_fn("train"), ...
0
votes
1
answer
112
views
tensorflow DenseHashTable lookup multi-dimensional keys
I want use DenseHashTable lookup string tensors, just like this answeranswer , keys' type is tf.string, value is embedding with tf.float32 dtype. But when keys is multi-dimensional, error occurs.
keys ...
1
vote
1
answer
80
views
Is there anyway to show the training progress from tf.estimator.LinearClassifier().train()
Is there any way to show the training progress from the TensorFlow linear estimator: tf.estimator.LinearClassifier().train() similar to how the progress output would be with a model.fit() for each ...
0
votes
0
answers
416
views
ImportError: cannot import name 'trace' from 'tensorflow.python.profiler'?
I installed the GPU-supported tesnorflow 2.2.0, on my machine, then I created a virtual enviorment using conda:
conda create --name tfgpu220 python==3.8
And I run these installation commands:
pip ...
1
vote
0
answers
149
views
Tensorflow 2 migrating estimators
There are some high level API's in Tensorflow 1 called Estimators. It's a collection of common ML algorithms. However, in the documentation it is stated that Estimators are not recommended for new ...
0
votes
1
answer
188
views
TensorFlow SavedModel output output had no dimensions when loading a model with BigQuery ML
I'm trying to use BigQuery ML to load a saved tensorflow model to make predictions.
However when I run the query that read the saved model in GCS I got the following error:
TensorFlow SavedModel ...
11
votes
2
answers
7k
views
Module 'tensorflow.tools.docs.doc_controls' has no attribute 'inheritable_header'
Running estimator from tensorflow today and came up with this error, any idea how to solve it?
File "C:\Users\ASUS Laptop\anaconda3\envs\tf_gpu\lib\site-packages\tensorflow_estimator\python\...
1
vote
0
answers
174
views
Why is keras slower than estimator in TensorFlow model trainning?
I trainning the same model through keras and estimator with TensorFlow in CPU, and found that keras was much slower than estimator.
batch_size = 256
emb_hash_bucket_size = 5000000
emb_dimension = 100
...
1
vote
1
answer
12k
views
AttributeError: module 'tensorflow_estimator.python.estimator.api._v1.estimator' has no attribute 'inpus'
I am trying to use linear classifier to predict, the constructing and training of the estimator is listed here:
model = tf.estimator.LinearClassifier(
n_classes = 2,
model_dir = "ongoing"...
1
vote
1
answer
902
views
OSError: [Errno 9] Bad file descriptor in tensorflow Estimater When deploying model on Multiple GPUs using tensorflow mirror strategy
I am trying to deploying deep learning model on two GPUs with single machine. I am utilizing TensorFlow mirror strategy. I am getting the following error:
Traceback (most recent call last):
Code
...
0
votes
1
answer
2k
views
Issue with tf.ParseExampleV2 when converting to Tensorflow Lite : "op is neither a custom op nor a flex op"
excuse my english.
I've been trying to handle Estimators API of tensorflow (v2.x), but when i'm trying to convert a model from tf.estimators to tflite with this code :
import tensorflow as tf
import ...
33
votes
8
answers
54k
views
ModuleNotFoundError: No module named 'tensorflow_core.estimator' for tensorflow 2.1.0
When using tensorflow, I have the following error messages
ERROR:root:Internal Python error in the inspect module.
Below is the traceback from this internal error.'
File "<frozen importlib....
1
vote
0
answers
195
views
Using feed_dict with Tensorflow Estimator API, "You must feed a value for placeholder tensor 'Placeholder' with dtype float and shape [?,784]"
I am trying to use a custom Estimator together with a feed_dict. Based on several relevant questions, such as this one, I have arrived at the following code. Note that I am returning dataset from ...
1
vote
1
answer
184
views
Using Tensorflow Estimators with Dataset API results in strange steps behavior
I'm facing some issues regarding the behavior of the training loop of Tensorflow's Estimator and Dataset APIs.
The code is as follows (tf2.3):
NUM_EXAMPLES = X_train.shape[0] # dataset has 8000 ...
0
votes
0
answers
102
views
tensorflow estimator passes train data through some weird normalization
Problem Description
I'm using tensorflow Estimator API, and have encountered a weird phenomenon.
I'm passing the exact same input_fn to both training and evaluation, and for some reason the images ...
1
vote
1
answer
852
views
early stopping in tensorflow object detection api
I am trying to implement early stopping in TF OD API. I used this code.
Here is my EarlyStoppingHook (is it essentially just a copy from the above code):
class EarlyStoppingHook(session_run_hook....
0
votes
1
answer
427
views
Two sets of shared embeddings from one tensorflow feature?
How can I create two sets of shared embeddings from the same tensorflow feature columns?
This small example
import tensorflow as tf
data = {"A": [0, 1, 2, 3], "B": [2, 1, 0, 3]}
...
1
vote
0
answers
142
views
FinalExporter not working in TensorFlow 2.1 on Google AI-Platform
I'm trying to upgrade my model to use AI-Platform 2.1 instead of 1.15, but I can't get the FinalExporter to work.
I followed the steps outlined in ai-platform: No eval folder or export folder in ...
2
votes
0
answers
357
views
How to perform custom data augmentation using Tensorflow Object Detection API and tf.estimator?
I'm aware that they are already provided in model.config by using something like:
data_augmentation_options {
random_adjust_contrast {
}
}
However, I need to perform my own data ...
0
votes
1
answer
136
views
Tensorflow Estimator: Samples using weighted distribution (probability)
I want to sample datas using weighted distribution (probability)
The examples are like below:
class distribution:
doc_distribution = {0: 40, 1: 18, 2: 8, 3: 598, ... , 9: 177}
I would to make the ...
0
votes
1
answer
330
views
Create Estimator from checkpoint and save as SavedModel without further training
I have created an Estimator from a TF Slim Resnet V2 checkpoint and tested it to make predictions. The main thing of what I did is basically similar to a normal Estimator together with ...
1
vote
1
answer
297
views
Hyperparameters tuning with tf.estimator.DNNClassifier
I've implemented the following model using the DNNClassifier class. The model is parametrized as follows
classifier = tf.estimator.DNNClassifier(
hidden_units=[60, 30, 20],
...
1
vote
2
answers
459
views
tf.data, construct a batch with different data?
I want to construct a batch of data with batchsize 16, using tf.data, where [:8] is a kind of data A, [8:16] is a kind of data B.
It is easy to do without tf.data. If use tf.data, the code could be:
...
0
votes
1
answer
563
views
Tensorflow 2: building an estimator with custom layer and tf.map_fn
I am trying to build an TensorFlow2 estimator from a custom Keras model. The model takes as input a tensor of shape [batch_size, n, h, w, c]. I need to apply a CNN on each [n, h, w, c] tensor from the ...
0
votes
1
answer
256
views
Making predictions with loaded Estimator in Tensorflow 2.0
I'm trying to follow this guide to load a SavedModel.
I first save the trained model as follows
feature_columns
>>> [NumericColumn(key='Fwd_IAT_Total', shape=(1,), default_value=None, dtype=...
2
votes
1
answer
78
views
TensorFlow1.15, the inner logic of Estimator's input_fn? Or the inner logic of MirroredStrategy?
I am pretraining BERT in 1 machine with 4 GPU, not 1 GPU.
For each training step, I am wondering whether the input_fn give 1 GPU 1 batch or give 4 GPU 1 batch.
The mirrow strategy code:
...
0
votes
1
answer
710
views
InvalidArgumentError: Input to reshape is a tensor with 0 values, but the requested shape has 54912
Very beginner question, I hope that's fine
I'm trying to train this model from GitHub with the MAPS dataset and I made new .tfrecords with this code for the train set. It is based from the code here ...
2
votes
1
answer
162
views
TensorFlow 2.1 using TPUEstimator: RuntimeError: All tensors outfed from TPU should preserve batch size dimension, but got scalar Tensor
I just converted an existing project from TF 1.14 to TF 2.1 which uses the TPUEstimator API. After making the conversion, testing locally (i.e. use_tpu=False) runs successfully. However, I am getting ...
6
votes
1
answer
1k
views
Tensorflow 2.0 - tf.estimator.DNNClassifier training for big datasets
I'm trying to train a DNNClassifier
labels = ['BENIGN', 'Syn', 'UDPLag', 'UDP', 'LDAP', 'MSSQL', 'NetBIOS', 'WebDDoS']
# Build a DNN
classifier = tf.estimator.DNNClassifier(
...
1
vote
1
answer
275
views
TensorFlow1.15, multi-GPU-1-machine, how to set batch_size?
I am pretraining BERT in 1 machine with 4 GPU.
The input function code:
def input_fn(params):
"""The actual input function."""
batch_size = FLAGS.train_batch_size
...
0
votes
1
answer
256
views
Tensorflow Estimator gives different prediction result on each call of predict
I trained a classifier for my own dataset using TF Estimators, but after each prediction call I'll get different prediction results. I checked the dataset and the data example order is alright , same ...
1
vote
0
answers
220
views
tf.estimator input_fn and eager mode
I tried to use numpy inside cnn_model.evaluate(), but it gave AttributeError: 'Tensor' object has no attribute 'numpy'. I used numpy to calculate accuracy and mean squared error using tf.keras.metrics....
0
votes
1
answer
178
views
tensorflow.estimator error: TypeError: 'builtin_function_or_method' object is not iterable
I can't seem to find the issue in the code that causes the error. I would appreciate it if you could help. Thanks in advance! The error generated from the code below is
TypeError: '...
0
votes
1
answer
193
views
AssertionError: `assert not _is_device_list_single_worker(devices)` while training estimator on ai-platform with mirrored strategy with TF 2.1
I'm running a training job on gcp ai-platform for tensorflow estimators with mirrored distribution strategy with --python-version 3.7 and --runtime-version 2.1.
I'm providing the necessary code ...
1
vote
0
answers
58
views
How to read and processing my image files in input_fn of Estimator in tensorflow? [duplicate]
I'm new in using TF. For some reasons, I have to use TF1.10, where I found that .ppm is not supported by tf.image.decode_image.
The final goal of my network is, to read a RGBD input, and use it to ...
0
votes
1
answer
255
views
Tensorflow Estimator Hook access to features labels passed to model_fn and graph operations built during model_fn
I am trying to understand a model built with the tensorflow Estimator framework. I'd like to use the Hooks API to add ops that process the input during evaluation, or prediction.
It seems I should ...
0
votes
1
answer
212
views
tensorflow estimator save model
I am Experimenting with Text Classification for character_rnn (https://github.com/tensorflow/tensorflow/blob/671baf080238025da9698ea980cd9504005f727c/tensorflow/examples/learn/...
1
vote
0
answers
480
views
Convert tensorflow 2.0 estimator to tensorflow lite
After exporting my saved_model.pb file, I would like to convert it in TfLite.
I had the same error than this post.
So I tried the proposed answer :
import tensorflow as tf
export_dir = './saved/...
1
vote
0
answers
258
views
What is the difference between an Estimator and a Regressor in TensorFlow tf.estimator?
TensorFlow Estimators (tf.estimator) are "high level tools for working with models". Instead of defining your own DNN classifier layer by layer there is already an estimator for that.
...
3
votes
1
answer
565
views
Performing inference with a BERT (TF 1.x) saved model
I'm stuck on one line of code and have been stalled on a project all weekend as a result.
I am working on a project that uses BERT for sentence classification. I have successfully trained the model, ...
0
votes
1
answer
275
views
Getting ReadTimeoutError for tensorflow while deploying python app to Azure Web App
remote: [20:44:36+0000] Collecting tensorflow==2.1.0
remote: [20:44:36+0000] Downloading tensorflow-2.1.0-cp37-cp37m-manylinux2010_x86_64.whl (421.8 MB)
remote: .......................................
0
votes
1
answer
503
views
Issue with TFX Trainer component not outputting model to filesystem
First of all, I am using TFX version 0.21.2 and Tensorflow version 2.1.
I have constructed a pipeline largely following the Chigaco taxi example. When the Trainer component is executed I can see the ...
1
vote
1
answer
149
views
How to determine the corresponding weight of values in a categorical feature while training a Linear Regression model in tensorflow?
I have trained a Linear regression model using tensorflow. I have both numerical and categorical features. For this example, one of the categorical features is "has_Alt", which takes a value of either ...
0
votes
0
answers
81
views
Suggestions on how to improve the accuracy
I am new dealing with the tf.estimators api. I have a Dnn classifier that is giving me an accuracy of 60% on the JM1 Software defect prediction dataset from PROMISE repository. I want to imporve the ...
0
votes
1
answer
200
views
Getting Precision and Recall is 0, where the accuracy is getting around 98%
I am working on DNNClassifier from Tensor flow estimation and used data set is JM1 (Defect Prediction )
Considered training features 0:8163(defects-free:6056, defects: 2106)
Considered validating ...
0
votes
1
answer
103
views
How to load a saved DNN estimator model for predicting?
I have a saved DNN estimator model. How can I restore the model for prediction?
I saved my model using:
#Create the input function
input_func = tf.estimator.inputs.pandas_input_fn(x=X_train, y=...
1
vote
1
answer
682
views
RecursionError in the Tokenization of a large database in TensorFlow
I have a lot of txt files, so I made a dataset following this tutorial of tensorflow:
https://www.tensorflow.org/tutorials/load_data/text?hl=es#build_vocabulary
My program works with smalls ...
4
votes
1
answer
699
views
Who to do early stopping with the evaluation loss using tf.estimator.train_and_evaluate?
I am using the Tensorflow estimator and explicitly the method tf.estimator.train_and_evaluate().
There is an early stopping hook for the training which is tf.contrib.estimator....