Skip to content

Commit 7a068ec

Browse files
tf-model-analysis-teamtfx-copybara
tf-model-analysis-team
authored andcommitted
Add nltk.download('punkt_tab') to Rouge metric.
This is needed to use the nltk.sent_tokenize() downstream. PiperOrigin-RevId: 686987036
1 parent 181ac2f commit 7a068ec

File tree

2 files changed

+2
-0
lines changed

2 files changed

+2
-0
lines changed

RELEASE.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@
1919
* Modifies a ROUGE Test to be compatible with NumPy v2.0.1.
2020
* Remove keras_util_test.py which is based on estimator models.
2121
* Remove dependency on eval_saved_model encodings.
22+
* Downloads `punkt_tab` in Rouge metric.
2223

2324
## Breaking Changes
2425

tensorflow_model_analysis/metrics/rouge.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,7 @@ def setup(self):
9393
if not tokenizer_installed:
9494
logging.info(_LOGGING_MESSAGE_TOKENIZER_PREPARER)
9595
nltk.download('punkt')
96+
nltk.download('punkt_tab')
9697

9798
def create_accumulator(self) -> _Accumulator:
9899
return _Accumulator()

0 commit comments

Comments
 (0)