Getting input from text file and storing into array but text file contains more than 20.000 strings-CodePudding

Getting inputs from a text file and storing it into an array but text file contains more than 20.000 strings. I'm trying to read strings from the text file and store them into a huge-sized array. How can I do that?

I can not use vectors. Is it possible to do it without using a hash table?

Afterward, I will try to find the most frequently used words using sorting.

CodePudding user response：

Assuming you're using C-Style / raw arrays you could do something like:

const size_t number_of_entries = count_entries_in_file();

//check if we actually have entries
assert(number_of_entries > 0);

std::string* file_entries = new std::string[number_of_entries];

//fill file_entries with the files entries
//...

//release heap memory again, so we don't create a leak
if(file_entries){
    delete[] file_entries;
    file_entries = nullptr;
}

CodePudding user response：

You do not need to keep the whole file in memory to count frequency of words. You only need to keep a single entry and some data structure to count the frequencies, for example a std::map<std::string,unsigned>.

Not tested:

std::map<std::string,unsigned> processFileEntries(std::ifstream& file) { 
    std::map<std::string,unsigned> freq;
    std::string word;
    
    while ( file >> entry ) {
                freqs[entry];
    }
    return freq;
}

For more efficient reading or more elaborated processing you could also read chunks of the file (eg 100 words), process chunks, and then continue with the next chunk.