Python | generator用于大文件读取

python生成器最常见的应用是大文件的读取，节省内存空间。

以下是使用生成器读取大文件的python代码：

def read_in_chunks(file_object, chunk_size=1024):
    """Lazy function (generator) to read a file piece by piece.
    Default chunk size: 1k."""
    while True:
        data = file_object.read(chunk_size)
        if not data:
            break
        yield data


def process_data(piece):
    print("piece of data is processing.")


f = open('really_big_file.dat')

for piece in read_in_chunks(f):
    process_data(piece)