I am following this guide on a rnn text generating network using tensorflow and numpy. I have been following the guide almost word for word but it has encountered an error on 23 and 26 due to a "TypeError: can only join an interable". I'm not sure what that could mean as I have been pretty much copying the guide. Anyone have an idea as to what the problem could be?
code:
import numpy
import sys
from nltk.tokenize import RegexpTokenizer
from nltk.corpus import stopwords
from keras.models import Sequential
from keras.layers import Dense, Dropout, LSTM
from keras.utils import np_utils
from keras.callbacks import ModelCheckpoint
file = open("C:/Users/alpha/Desktop/coding/pickles.txt").read()
def tokenizeWords(input):
# lowercase everything to standardize
input = input.lower()
# instantiate the tokenizer
tokenizer = RegexpTokenizer(r'w+')
tokens = tokenizer.tokenize(input)
# if the created token isn't in the stop words, make it part of "filtered"
filtered = filter(lambda token: token not in stopwords.words('english'), tokens)
return "".join(filter)
processed_inputs = tokenizeWords(file)
chars = sorted(list(set(processed_inputs)))
char_to_num = dict((c, i) for i, c in enumerate(chars))
input_len = len(processed_inputs)
vocab_len = len(chars)
print("Total number of characters", input_len)
print("Total vocab", vocab_len)
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…