Search

18.7 — Random file I/O

The file pointer

Each file stream class contains a file pointer that is used to keep track of the current read/write position within the file. When something is read from or written to a file, the reading/writing happens at the file pointer’s current location. By default, when opening a file for reading or writing, the file pointer is set to the beginning of the file. However, if a file is opened in append mode, the file pointer is moved to the end of the file, so that writing does not overwrite any of the current contents of the file.

Random file access with seekg() and seekp()

So far, all of the file access we’ve done has been sequential -- that is, we’ve read or written the file contents in order. However, it is also possible to do random file access -- that is, skip around to various points in the file to read its contents. This can be useful when your file is full of records, and you wish to retrieve a specific record. Rather than reading all of the records until you get to the one you want, you can skip directly to the record you wish to retrieve.

Random file access is done by manipulating the file pointer using either seekg() function (for input) and seekp() function (for output). In case you are wondering, the g stands for “get” and the p for “put”. For some types of streams, seekg() (changing the read position) and seekp() (changing the write position) operate independently -- however, with file streams, the read and write position are always identical, so seekg and seekp can be used interchangeably.

The seekg() and seekp() functions take two parameters. The first parameter is an offset that determines how many bytes to move the file pointer. The second parameter is an Ios flag that specifies what the offset parameter should be offset from.

Ios seek flag Meaning
beg The offset is relative to the beginning of the file (default)
cur The offset is relative to the current location of the file pointer
end The offset is relative to the end of the file

A positive offset means move the file pointer towards the end of the file, whereas a negative offset means move the file pointer towards the beginning of the file.

Here are some examples:

Moving to the beginning or end of the file is easy:

Let’s do an example using seekg() and the input file we created in the last lesson. That input file looks like this:

This is line 1
This is line 2
This is line 3
This is line 4

Here is the example:

This produces the result:

is line 1
line 2
his is line 4

Note: Some compilers have buggy implementations of seekg() and tellg() when used in conjunction with text files (due to buffering). If your compiler is one of them (and you’ll know because your output will differ from the above), you can try opening the file in binary mode instead:

Two other useful functions are tellg() and tellp(), which return the absolute position of the file pointer. This can be used to determine the size of a file:

This prints:

64

which is how long sample.dat is in bytes (assuming a carriage return after the last line).

Reading and writing a file at the same time using fstream

The fstream class is capable of both reading and writing a file at the same time -- almost! The big caveat here is that it is not possible to switch between reading and writing arbitrarily. Once a read or write has taken place, the only way to switch between the two is to perform an operation that modifies the file position (e.g. a seek). If you don’t actually want to move the file pointer (because it’s already in the spot you want), you can always seek to the current position:

If you do not do this, any number of strange and bizarre things may occur.

(Note: Although it may seem that iofile.seekg(0, ios::cur) would also work, it appears some compilers may optimize this away).

One other bit of trickiness: Unlike ifstream, where we could say while (inf) to determine if there was more to read, this will not work with fstream.

Let’s do a file I/O example using fstream. We’re going to write a program that opens a file, reads its contents, and changes any vowels it finds to a ‘#’ symbol.

Other useful file functions

To delete a file, simply use the remove() function.

Also, the is_open() function will return true if the stream is currently open, and false otherwise.

A warning about writing pointers to disk

While streaming variables to a file is quite easy, things become more complicated when you’re dealing with pointers. Remember that a pointer simply holds the address of the variable it is pointing to. Although it is possible to read and write addresses to disk, it is extremely dangerous to do so. This is because a variable’s address may differ from execution to execution. Consequently, although a variable may have lived at address 0x0012FF7C when you wrote that address to disk, it may not live there any more when you read that address back in!

For example, let’s say you had an integer named nValue that lived at address 0x0012FF7C. You assigned nValue the value 5. You also declared a pointer named *pnValue that points to nValue. pnValue holds nValue’s address of 0x0012FF7C. You want to save these for later, so you write the value 5 and the address 0x0012FF7C to disk.

A few weeks later, you run the program again and read these values back from disk. You read the value 5 into another variable named nValue, which lives at 0x0012FF78. You read the address 0x0012FF7C into a new pointer named *pnValue. Because pnValue now points to 0x0012FF7C when the nValue lives at 0x0012FF78, pnValue is no longer pointing to nValue, and trying to access pnValue will lead you into trouble.

Rule: Do not write addresses to files. The variables that were originally at those addresses may be at different addresses when you read their values back in from disk, and the addresses will be invalid.

A.1 -- Static and dynamic libraries
Index
18.6 -- Basic file I/O

90 comments to 18.7 — Random file I/O

  • cdecde57

    I googled how to put all a files contents into a string and I found a way and I implemented it into my program like this.

    the scramble function will take the string with the key and encrypt it. The 1 just means that it is decrypted otherwise it would encrypt.

    It works AMAZINGLY WELL

    BUT for some reason, music and video files do not work. I literally made a 35mb text file and it encrypted and decrypted it just fine but video and music files do not? Do you think that is because it has unsupported characters in them, and if so what do I do?

    I did change the program a bit but do you know anywhere where I can learn more about the fstream? I love using it but I don't understand everything here like the std::istreambuf_iterator or
    std::string str(static_cast<std::string::size_type>(sSize), '\0');

    which I know it is a conversion of sSize to the std::string::size_type and that \0 is the null value to like stop it.

    Anyways this site is the only place I have gone to learn c++ and currently I am in unit 8 learning classes but skipped here real fast to learn about fstream. I love this site and how you teach I am just wondering if you know a good place to learn more about fstream and things?

    Thanks!

    • Music and video files have 0-valued bytes. Strings are 0-terminated, so they stop when they find a 0. You need to either allocated a buffer yourself or use an `std::vector`. I guess you'll have to pause your project here and continue the lessons where you left off.

  • cdecde57

    Hello! I need some help with this program. This program will encrypt or decrypt files. But the problem is that if you have whitespace like an enter it will not work. I made a database a while ago that works amazingly well so I have a general idea on what I'm doing but the problem is that I have to scan each value of the string to change its value to encrypt or decrypt it. Because of this, I use an if statement and it does not scan the file properly. It is a problem with how it is encrypting/decrypting. Here is the program.

    I have considered using a c-style string because I think that it might work. But I am wondering if you have any ideas.

    By the way the std::couts are really big and are bigger than 80 characters normally just so you know.

    btw the cls(); function I call is just a function that spams a ton of \n to clear the screen. Here it is if you want it.

    I don't know what to do for the encrypting / decrypting. Please help.

    If you need the libraries these are the ones I have in it. Some I don't use but just put in anyways in case i do sometime in the development.

    • What I saw on a quick glance
      - Initialize your variables with brace initializers.
      - Limit your lines to 80 characters in length for better readability on small displays.
      - Re-read the lesson on switch-statements, especially fall-through.
      - Don't use `exit`.
      - Use `std::string(123, '\n')` to repeat a character.
      - Using `main` produces undefined behavior. I'd be surprised if your compiler didn't warn you.
      - You're under/overflowing your chars. Make sure they don't leave the range [0, 255].
      - `|` is not the same as `||` (Chapter O).

      Whenever you need to preserve the original format, open the file in binary mode. Using string extraction functions will almost certainly remove some white space that you wanted to keep. Read the entire file at once.

      • cdecde57

        So I am trying to do that but the

        ifs.read gives me an error saying it was const.

        Just to be clear this will copy all the files contents to a string or something right?

        Also, I was just wondering where I am overflowing the char so I can fix it. If it is where I am encrypting the strings values I do intend that but if it is another place than that would be nice to know where.

        On the switch statments on fall - through isn't that when I don't say break; at the end of a case and it makes it go into the others?

        Thanks!

        • > ifs.read gives me an error saying it was const
          I can't help you without your code and exact error message.

          > this will copy all the files contents to a string
          Correct

          > where I am overflowing the char
          Line 225, 268. Overflowing numbers yields undefined behavior. Perform your calculation using `int`, then convert the result to a `char` using the modulus operator (`%`).

          > isn't that when I don't say break
          Correct. Since your cases do the same, you should let them fall-through instead of duplicating your code.

  • Hexadecimal255

    Its a sudoku field which should generate a random puzzle and my problem is it only works 40% a time. :(
    It probably ends in a infinite loop in these scenarious where the programm wont finish correctly
    but i dont really understand why.
    When I delete/comment out the line with // PROBLEMATIC:...
    it works, but I need it otherwise it will generate less numbers than needed.

  • Richard

    I have used C-language random file I/O for years, and after going through Alex’s excellent tutorial, I thought I would try C++ <fstream>.  However, when I open an fstream object (std::fstream fileh) to write in binary,

    a call to operator<< does NOT appear to write an int in binary format.  After writing an int, the file position is not incremented by sizeof(int) but is incremented according to the ASCII conversion.

    If I use the fstream member function write, fstream wants a char * for the buffer, and gives a type error for any other pointer type; that is,

    where intbuffer is an integer array, gives an error.  Same if &intbuffer[0] is replaced with intbuffer or a variable of type int *.

    Unlike fwrite() in <cstdio>, fstream .write() won’t accept any pointer type except char *, meaning one has to cast int * to void *, and then void * to char *, all of which seems a bit excessive.

    Any comments?  And does anyone have any experience with, and wish to comment on, streambuf?

    • The binary flag only means that the data is not allowed to be modified by the implementation, ie. what you read is what you wrote.
      @operator<< still outputs as text.
      You can use an @reinterpret_cast to perform type-unsafe pointer casts.

      Or create a custom ofstream object that overrides @operator<<.

      If you do this, keep in mind that arrays decay to pointers. You'll need another overload of @operator<< to print arrays, or print the array in a loop.

      • Richard

        nascardriver, thanks so much for your post and quick solutions!

        This occasion is a justifiable use of reinterpret_cast, which I tend to avoid (like the plague) due to its loss of original pointer type and system-specific implications.  Although void * recasting really has similar implications...

        Since my original post I've implemented <fstream> member functions .read() and .write() and they work like a charm.

        Pardon my (really) old school, but my first impression on encounter of a "binary" attribute for a newly-learned file system is that a variable's data is written byte-by-byte for exactly the same size and Endianness as the binary representation in memory.

  • Minh Quan

    Hi Alex,

    I want input data from file ms word (*.doc or *.docx) and output data in same file ms word. Please help me how to make it. Thank you so much.

  • Ramesh

    I did not understand the need of following line in last program
    iofile.seekg(iofile.tellg(), ios::beg);

    Program executes correctly without that line.

    • Alex

      Re-read the lesson, it tells you why this is needed. Your compiler may have some non-standard behavior that causes the program to work without this line, but that doesn't mean you should omit it.

      • Ramesh

        Hi Alex, thanks for the reply.

        There is comment in program.
        We'll seekg() to the current location because we don't want to move the file pointer.
        >> Read/write operation happens at current location of file pointer, then what is need to move file pointer to current location using seekg() ?

        • Alex

          From the article: "The fstream class is capable of both reading and writing a file at the same time -- almost! The big caveat here is that it is not possible to switch between reading and writing arbitrarily. Once a read or write has taken place, the only way to switch between the two is to perform an operation that modifies the file position (e.g. a seek)".

          We can use a seek to the current file position to tell the class we're going to switch between read and write.

      • Hi Alex!

        > it is not possible to switch between reading and writing arbitrarily
        The only source that backs this statement up is the documentation of @fopen (from C). I cannot find anything referring to this behavior in documentations for any of the C++ file access functions, nor in the standard. Since @std::fstream has a shared buffer and shared pointer for read and write access, I don't understand why switching between read and write should cause problems.
        Please link trustworthy sources or tell me that you found a compiler which causes undefined behavior when the line in question is omitted so I have a reason to keep on searching.

        • Alex

          Hey Nascardriver. Thanks for keeping me honest.

          I see a lot of anecdotal reports about this, but very little in the way of official sources. Two notes:
          1) Stanley Lippman reports about this in his book: https://books.google.com/books?id=J1HMLyxqJfgC&pg=PT1391&lpg=PT1391&dq=c%2B%2B+fstream+switch+read+write+seek&source=bl&ots=FJQEUi2bcq&sig=lIUs6SaV2xP_Nk1ITs2qYT139No&hl=en&sa=X&ved=2ahUKEwilkqTy5sncAhViKH0KHUDWBuQQ6AEwCXoECAkQAQ#v=onepage&q=c%2B%2B%20fstream%20switch%20read%20write%20seek&f=false -- I learned C++ from Lippman books, he's very well regarded.
          2) Visual Studio 2017 requires this line, otherwise it doesn't work, so that's at least one compiler exhibiting this behavior.

          If you can find a more official source, I'd love to know about it.

          • Thanks for getting back to me.
            Thanks to Mike Kinghan over at stackoverflow I've found the relevant information.
            I cannot describe it any better than he did, here's the link https://stackoverflow.com/a/17567454/9364954

            TL;DR: You're right, the call is required. The GNU C library diverges from the standard, allowing your code to work without the call to @std::fstream::seekg.

            • Alex

              Fantastic! Thanks for digging this up.

              I should probably add a general note to the preamble of these tutorials about non-conformant compiler behaviors (of which there are a lot), since those tend to drive a lot of questions/comments ("I did bad thing X or didn't do required thing Y and it still worked").

  • Michael

    Hi Alex,
    why did you do a seekg() when you want to write something in? Should we use seekp() then?

    • Alex

      For file streams, there's only one file pointer, so seekg() and seekp() operate identically. It's probably slightly more correct to use seekp() when intending to write, but functionally it doesn't make a difference in this case.

  • Luhan

    Thanks for the awesome tutorial alex, you did save me when I was looking for a place to learn c++. If you let me ask you a last question, what book would you think would fit more for me right now between "The C++ Standard Library: A Tutorial and Reference" (which you did indicate) and "Effective Modern C++", I'll start learning SFML.

    • Alex

      They're totally different books. The C++ Standard Library book is meant to teach you about the standard library in a lot more detail. Effective Modern C++ is meant to teach you how to use some of C++'s more advanced features. Personally, I think you'll get more mileage out of the C++ Standard Library book, as it's always better to reuse what already exists than build new things yourself if you can.

  • Satwant

    Hi Alex,
             I am having problem saving dynamically allocated array into files. While adding static arrays to file works fine, when storing dynamic arrays the program starts to crash..
    Is there a different way to store dynamic allocated data to files? I have a class with dynamic array and i am storing data to file with following commands:

                                     Please Help.

    • Alex

      Assuming that t1 is a pointer, sizeof(t1) would be the size of the pointer. What you actually want is the size of the whole array. You can do this by multiplying the array length by the size of the first element (*t1).

  • Raj

    Hello Alex, firstly thank you for these tutorials i recently completed it.
    can you please suggest me a project than can be done using the knowledge i have attained here, and would also add value to my resume

    Raj

    • Alex

      Not really. Think about what kind of problems would be interesting to you to solve, and then assess whether what you know is sufficient to build said solution.

Leave a Comment

Put all code inside code tags: [code]your code here[/code]