Conditional random fields in PyTorch.
- Python 3.6
- PyTorch 1.0.0
Install with pip:
pip install pytorch-crf
Or, install from Github for the latest version:
pip install git+https://github.com/kmkurn/pytorch-crf#egg=pytorch_crf
>>> import torch >>> from torchcrf import CRF >>> num_tags = 5 # number of tags is 5 >>> model = CRF(num_tags)
Computing log likelihood¶
Once created, you can compute the log likelihood of a sequence of tags given some emission scores.
>>> seq_length = 3 # maximum sequence length in a batch >>> batch_size = 2 # number of samples in the batch >>> emissions = torch.randn(seq_length, batch_size, num_tags) >>> tags = torch.tensor([ ... [0, 1], [2, 4], [3, 1] ... ], dtype=torch.long) # (seq_length, batch_size) >>> model(emissions, tags) tensor(-12.7431, grad_fn=<SumBackward0>)
If you have some padding in your input tensors, you can pass a mask tensor.
>>> # mask size is (seq_length, batch_size) >>> # the last sample has length of 1 >>> mask = torch.tensor([ ... [1, 1], [1, 1], [1, 0] ... ], dtype=torch.uint8) >>> model(emissions, tags, mask=mask) tensor(-10.8390, grad_fn=<SumBackward0>)
Note that the returned value is the log likelihood so you’ll need to make this value
negative as your loss. By default, the log likelihood is summed over batches. For other
options, consult the API documentation of
Conditional random field.
This module implements a conditional random field [LMP01]. The forward computation of this class computes the log likelihood of the given sequence of tags and emission score tensor. This class also has
decodemethod which finds the best tag sequence given an emission score tensor using Viterbi algorithm.
Parameters: [LMP01] Lafferty, J., McCallum, A., Pereira, F. (2001). “Conditional random fields: Probabilistic models for segmenting and labeling sequence data”. Proc. 18th International Conf. on Machine Learning. Morgan Kaufmann. pp. 282–289.
Find the most likely tag sequence using Viterbi algorithm.
Parameters: Return type: Returns:
List of list containing the best tag sequence for each batch.
forward(emissions, tags, mask=None, reduction='sum')¶
Compute the conditional log likelihood of a sequence of tags given emission scores.
- emissions (
Tensor) – Emission score tensor of size
(seq_length, batch_size, num_tags)if
(batch_size, seq_length, num_tags)otherwise.
- tags (
LongTensor) – Sequence of tags tensor of size
- mask (
ByteTensor) – Mask tensor of size
- reduction (
str) – Specifies the reduction to apply to the output:
none: no reduction will be applied.
sum: the output will be summed over batches.
mean: the output will be averaged over batches.
token_mean: the output will be averaged over tokens.
The log likelihood. This will have size
(batch_size,)if reduction is
- emissions (
Initialize the transition parameters.
The parameters will be initialized randomly from a uniform distribution between -0.1 and 0.1.