1,721 questions
Advice
0
votes
0
replies
29
views
When using TensorDictPrioritizedReplayBuffer, should I apply the priority weight manually or not?
With Prioritized Experience Replay (PER), we use Beta parameter, so we can find weight that will be used to offset the bias introduced by PER. Now, with PyTorch's TensorDictPrioritizedReplayBuffer, I ...
- 13
0
votes
0
answers
61
views
How do I compute validation loss for a fine-tuned Qwen model in Hugging Face Transformers during evaluation?
I trained a Qwen model on my own dataset. Now I need to evaluate my trained model using the loss function, but I don’t know how to do it. I saw examples for other metrics such as accuracy and ...
1
vote
1
answer
173
views
Multiclass focal loss in xgboost doesn't train
I have a dataframe with 60 columns as variables and a last column as target class (4 possible classes). I want to implement a custom loss function. I want that function to be the focal loss for a ...
- 1,297
2
votes
1
answer
112
views
How to handle loss function with sparse output
I'm trying to create a ML model in TensorFlow that takes in a tensor with shape (128,128,12) and outputs a tensor with shape (128,128,3), where the output dimensions mean (x, y, sensor_number).
With ...
- 731
6
votes
1
answer
468
views
Constructing custom loss function in lightgbm
I have a pandas dataframe that records the outcome of F1 races:
data = {
"Race_ID": [0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4],
"Racer_Number": [1, 2, 3, ...
- 592
0
votes
0
answers
105
views
My testing loss isn't improving while I'm using same train and test data
I'm trying to fine tune a model for a segmentation task. To see if everything is working properly I'm trying to make my model overfit on a single image split in 16 different patches.
So in my code, I ...
- 331
0
votes
0
answers
50
views
How to update model weights from a custom arbitrary loss function in Pytorch?
Starting with the end in mind:
Is there anyway I can write that arbitrary loss function by calling some pytorch functions, thus preserving the autograd graph?
How can I ensure my loss function is &...
-1
votes
1
answer
58
views
implement a differentiable L0 regularizer to keras layer
What is the appropriate way to implement a differentiable variant of L0 regularizer (count the non-zero values in a Conv layer / matrix) to keras layer?
I was thinking of using r(x) = tanh(abs(f*x)) ...
- 170
1
vote
1
answer
92
views
LogVar layer of a VAE only returns zeros
I'm building a Variational auto encoder (VAE) with tfjs.
For now I'm only exploring with the fashionMNIST dataset and a simple model as follows:
input layer (28*28*1)
flatten
intermediate_1 (dense 50 ...
- 13
0
votes
0
answers
89
views
Custom loss function in XGBoost
I would like to create a custom loss function for the "reg:pseudohubererror" objective in XGBoost. However, I am noticing a discrepancy between the results produced by the default "reg:...
- 1
1
vote
0
answers
47
views
MLP with data and physics loss
I have the following problem: Training an MLP on 4 inputs while also having an estimation from a physical model (with some error). I now want to compute a combined loss from the physics and data loss ...
0
votes
0
answers
34
views
How to change last layer in finetuned model?
When I fine-tuned the model Hubert to detect phoneme, I chose a fine-tuned ASR Hubert model and I removed the last two layers and added a linear layer to the config vocab_size of phoneme. What is ...
- 1
2
votes
0
answers
26
views
Fitting Honeycomb Template to Grid of Detected Honeycomb Points (Different Number of Points in Each)
Here is some setup for my problem. I am dealing with greyscale images from a pinhole camera. The pinholes are in a honeycomb pattern that can have a rotation, scale, and translation but by small ...
- 21
0
votes
0
answers
40
views
MATLAB Neural Network Predictions Converging to 1
I am trying to train a neural network but I am hitting a snag. I have a simplified version of the problem I am facing:
Setup:
NN = trainnet(X_data, Y_data, NN, 'crossentropy', options);
X_data is ...
- 99
0
votes
1
answer
19
views
Global minimum as a starting point of Gradient Descent
If I already have the Global Minimum value for the Cost function of any model (including large language models) - would it facilitate Gradient Descent calculation?
(suppose I have a quick way to ...
- 355
0
votes
0
answers
78
views
100% accuracy for multi_class classification
I am training a model for multi label classification task for each class I have multiple labels after running the test i got 100% for both classes
I used 150 000 images for training and validation and ...
- 19
0
votes
0
answers
90
views
Keras loss function for horse racing - profitability
I am currently developing a Keras model that can more accurately predict the outcomes of Australian horse races.
The current loss function incorporates a Brier skill score and is consistently more ...
- 41
0
votes
0
answers
67
views
Correct loss function for bboxes in a detector model
I try to clarify the learning process of the detector model with anchors.
Unfortunately, I have some trouble with the loss function. I have built the model with the classification and regression heads,...
- 11
2
votes
1
answer
51
views
How do I update pixelClassificationLayer() to a custom loss function?
I have seen in the Mathworks official website for the pixelClassificationLayer() function that I should update it to a custom loss function using the following code:
function loss = modelLoss(Y,T)
...
- 23
2
votes
0
answers
74
views
Can't replicate the behaviour of loss calculation in keras
I am working on a semantic segmentation problem where I have an input x ( CT image) to a deep learning model of shape (batch_size,1,256,256) and an output of shape (batch_size,2,256,256) where the ...
- 105
0
votes
1
answer
40
views
Linear regression model barely optimizes the intercept b
I've programmed a linear regression model from scratch. I use the "Sum of squared residuals" as the loss function for gradient descent. For testing I use linear data (y=x)
When running the ...
-1
votes
1
answer
96
views
Optimizing Physics Informed Neural Network
I am trying to solve 2D wave equation using physics informed neural network (PINN). I have total four terms in my loss function. I faced two issues while training my model. The total loss drops really ...
0
votes
0
answers
135
views
How should I properly use KID Score (FID Score)
for my final master degree project I am trying to do Data Augmentation in a dataset of thermal images (black and white) to detect breast cancer. This datasets contains only 280 images, that can be ...
- 43
1
vote
1
answer
381
views
Contrastive Loss from Scratch
I am trying to implement/learn how to implement contrastive loss. Currently my gradients are exploding into infinity and I think I must have misimplemented something. I was wondering if someone could ...
- 37
1
vote
0
answers
42
views
problem with brain tumor segmentation 2D Unet with Dice_coef_loss
Knowing that I am training using the 4 MRI modalities, when I use categorical cross-entropy, in this tutorial from brain_tumor_segmentation_u_net the IOU and Dice coefficients work fine. However, when ...
- 19
-1
votes
1
answer
50
views
pytorch s3fd pga_attack, problem in loss.backward() to get grad.data [closed]
from detection.sfd import sfd_detector
def pgd_attack(model, input_data, eps=0.03, alpha=0.01, attack_steps=1, device='cpu'):
ta = input_data.requires_grad_(True).to(device)
perturbation = ...
- 3
1
vote
1
answer
177
views
Why to use combined loss function for segmentation and classification [closed]
I tried to modify the U-NET model for one-dimensional data by providing an additional branch, attached to the last encoding block, whose purpose is to classify data into eighteen areas of the cortex. (...
- 13
1
vote
0
answers
25
views
Zero out loss for a certain pixel class in BCELossWithLogits
I am performing binary semantic segmentation on a dataset of pets (dogs and cats) each pixel has a class. There are 3 classes, foreground(1.0), background(0.0) and unclassified pixels(0.5020). I only ...
- 11
0
votes
0
answers
42
views
How to make a customloss function with more variable inputs work
I am making or trying to make a model that can estimate intrinsic camera parameters. For this, I want to use a custom loss function that uses the estimated intrinsic camera parameters, and then ...
0
votes
1
answer
93
views
How can I make a custom loss function in tensorflow, which takes inputs of model, predictions of model, and weights of model as it's parameters
I am trying to make a model which solves a partial differential equation (PDE). The problem is, it requires a make loss function which takes as its parameters:
Inputs given to model
Predictions of ...
- 11
0
votes
1
answer
130
views
How do I handle a custom loss function with (1/(1-exp(-x))-1/x) in it?
I am working on a deep learning model with a ragged tensor where the custom loss function is related to:
f(x)+f(x+50)
and f(x)=1/(1-exp(-x))-1/x when x!=0, f(x)=0.5 when x=0.
f(x) is ranged between 0 ...
- 15
0
votes
1
answer
61
views
TensorFlow: Calculating gradients of regularization loss terms dependent on model input and output
Overview
My model is an encoder that has input Z and output x.
I'm trying to use a total_loss that has both traditional supervised learning and regularization term(s). I have additional functions (...
- 13
0
votes
0
answers
101
views
PyTorch loss.backward() Error: TypeError: 'NoneType' object is not callable
I encountered an error while training a Siamese network using PyTorch. When I call loss.backward() after calculating the loss, I receive a type error. Below are the error messages and the relevant ...
1
vote
1
answer
130
views
In Keras, how can I save and load a neural network model that includes a custom loss function?
I am having difficulty saving and reloading a neural network model when I use a custom loss function. For example, in the code below (which integrates the suggestions of the related questions here ...
- 427
0
votes
0
answers
90
views
Binary Crossentropy on KerasTensors not working
I'm trying to implement this VAE in tf/keras and there seems to be something wrong with binary_crossentropy.
import tensorflow as tf
from tensorflow.keras import backend as K
from tensorflow.keras....
- 439
1
vote
0
answers
38
views
How to use Encoder predictions for additional loss and gradient calculations (Tensorflow)
Problem
I'm having troubles correctly adding physics-informed losses to my training code for my neural network.
Background
I have an encoder that takes an input curve, X(w), where w is an independent ...
- 13
0
votes
1
answer
50
views
Multi loss going into the same subsquent model using PyTorch
I am currently facing a problem.
I am wondering how I should back propagate the loss functions in the following model.
What is important here is that all the blue part is common to the 2 outputs, the ...
- 78
-1
votes
1
answer
56
views
In UNet3+: RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation
when i try to use the UNet3+ i get an error:
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [1, 1024, 64, 64]], ...
- 21
0
votes
0
answers
34
views
value error "No gradient provided for any variable", in custom loss function
I am fine-tuning an already trained model using a custom loss function. I am working on an audio related problem. When my input file has speech then the gradients are being calculated effectively for ...
1
vote
0
answers
27
views
"No gradients were provided for any variables" while making an mse+round+rescale loss function
Problem Statement
I have a TensorFlow machine learning model that should learn to mimic a digital result [0, 1], the output of the model is a float number, so I have used a MeanSquaredError and ...
- 41
0
votes
0
answers
72
views
tensorflow>=2.16.2 -> compile doesn't use provided loss function
I am training a model with tensorflow 2.x.
Up to now, I was using 2.14.1 and I had no issue. Now, I upgraded to 2.16.2 (also tried 2.17.0) and the same code doesn't work anymore. The whole model ...
- 57
1
vote
0
answers
32
views
Is there a way to train a CNN with the loss of another net?
I have 2 CNNs. One is called net and the other is called mask. Both have the same architecture but are initialised differently. My training loop looks like this:
for epoch in range(epochs):
print(...
-2
votes
1
answer
53
views
Which loss function should be used if sum(y_true)=1?
My yTrue are basically like [.2,.8] but never [1,0] or [0,1]
sum(yTrue)=1 always
I tried CategoricalCrossentropy But a TypeError occured-
TypeError: Expected float32, but got <keras.src.losses....
0
votes
1
answer
49
views
Inconsistent results between PyTorch loss function for `reduction=mean`
In particular, the following code block compares using
nn.CrossEntropyLoss(reduction='mean') with loss_fn = nn.CrossEntropyLoss(reduction='none')
followed by loss.mean().
The results are surprisingly ...
- 484
2
votes
0
answers
205
views
Objectness/IoU loss computation in YOLOX model
I've coded YOLOX from sctrach, but I have a problem with the objectness loss (IoU branch). In the paper they not report how the loss is computed. However, searching online I found this formula
YOLOX ...
0
votes
1
answer
119
views
Keras 3 Custom Loss Function to mask NaN
I am trying to build a custom Loss function on Keras 3, which would be used either in jax or torch backend.
I want to mask out of y_pred and y_true all the indices where y_true is a certain value. ...
- 23
1
vote
0
answers
46
views
Loss is Nan with Tensorflow
I've developed a TensorFlow model for an artificial intelligence project, but I'm having a problem with NaN in the loss function during training. Here's an extract from my code:
import os
os.environ['...
1
vote
1
answer
67
views
How to calculate loss over a sliding window of samples and then backpropagate the weighted average loss
I am trying to implement a learning technique from a paper. The relevant portion is: The SNN baseline used a sliding window of 50 consecutive data points, representing 200 ms of data (50-point window, ...
0
votes
0
answers
21
views
Using pytorch, how to estalish a loss function to keep all kernal to be exactly same
For exmaple, you have a CNN of 3 layers, every layer has only 1 channal(1 output features), so the weights of the CNN are 3 kernals, 1 for each layer. Assume these 3 kernals all are 221, and how can ...
1
vote
0
answers
282
views
How to define the forward pass in a custom PyTorch FasterRCNN?
I'm trying to write a custom PyTorch FasterRCNN that's based on an existing PyTorch model, fasterrcnn_resnet50_fpn, but I'm getting stuck on how to correctly write the forward pass. Following this ...
- 1,327