C# IEnumerable and yield return: How foreach Actually Works

The extension methods post ended with a hand-rolled Where containing a keyword I did not explain: yield return. That keyword is how LINQ is lazy, and IEnumerable<T> is the interface every LINQ operator is written against. This post takes the lid off both. By the end you will know exactly what foreach compiles into, why a LINQ query can filter a file bigger than your RAM, and why returning IEnumerable from a repository method is sometimes a bug.

What foreach actually is

Start with the loop everyone writes:

foreach (var book in books)
    Console.WriteLine(book.Title);

foreach is not a language primitive over lists. It is sugar for this:

var enumerator = books.GetEnumerator();
try
{
    while (enumerator.MoveNext())
    {
        var book = enumerator.Current;
        Console.WriteLine(book.Title);
    }
}
finally
{
    (enumerator as IDisposable)?.Dispose();
}

That is the whole mechanism, and it explains a lot. The compiler asks the collection for an enumerator, then repeatedly calls MoveNext() and reads Current. Nothing about a count, an index, or an array. foreach never asks “how many are there” and never asks for item number seven; it only ever asks “is there another one” and “what is it”.

Two interfaces describe that contract:

public interface IEnumerable<T>          // "you can iterate me"
{
    IEnumerator<T> GetEnumerator();
}

public interface IEnumerator<T>          // "I am a cursor over a sequence"
{
    T Current { get; }
    bool MoveNext();
    void Reset();
}

IEnumerable<T> is the sequence. IEnumerator<T> is the cursor walking it. A collection can hand out many independent enumerators, which is why two nested foreach loops over the same list do not interfere.

This is also why an interface with one method supports fifty LINQ operators, as the last post explained: Where needs nothing more than “give me the next item”.

Writing an enumerator by hand, once

To appreciate yield return, write the cursor manually one time. Suppose we want a sequence of books priced over a threshold:

public class ExpensiveBooks : IEnumerable<Book>
{
    private readonly List<Book> _source;
    private readonly int _threshold;

    public ExpensiveBooks(List<Book> source, int threshold)
    {
        _source = source;
        _threshold = threshold;
    }

    public IEnumerator<Book> GetEnumerator() => new Cursor(_source, _threshold);
    IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();

    private class Cursor : IEnumerator<Book>
    {
        private readonly List<Book> _source;
        private readonly int _threshold;
        private int _index = -1;

        public Cursor(List<Book> source, int threshold)
        {
            _source = source;
            _threshold = threshold;
        }

        public Book Current => _source[_index];
        object IEnumerator.Current => Current;

        public bool MoveNext()
        {
            while (++_index < _source.Count)
            {
                if (_source[_index].Price > _threshold)
                    return true;
            }
            return false;
        }

        public void Reset() => _index = -1;
        public void Dispose() { }
    }
}

Forty lines, a nested class, a hand-maintained _index field, and two interface implementations, to express “give me the books over a threshold”. Notice what the class is really doing: it is remembering where it got to between calls. _index exists purely so MoveNext can resume.

yield return writes that class for you

public static IEnumerable<Book> ExpensiveBooks(List<Book> source, int threshold)
{
    foreach (var book in source)
    {
        if (book.Price > threshold)
            yield return book;
    }
}

Six lines, and the compiler generates the state machine class above, _index and all. That is not an analogy: put this in a project, open the assembly in a decompiler, and you will find a generated nested class implementing IEnumerator<Book> with a <>1__state field.

A method containing yield return is called an iterator method. Two rules define its behaviour, and everything else follows:

Rule one: the body does not run when you call it. Calling ExpensiveBooks(books, 10) executes not a single line of the body. It constructs the state machine and returns it immediately. This is the deferred execution from the LINQ post: now you know the mechanism behind it.

Rule two: yield return pauses the method, it does not exit it. When the consumer calls MoveNext(), the body runs until it hits a yield return, hands that value over as Current, and freezes: locals, loop position, everything preserved. The next MoveNext() resumes on the line after the yield return. It is a return that comes back.

You can watch it happen:

public static IEnumerable<int> Counting()
{
    Console.WriteLine("start");
    yield return 1;
    Console.WriteLine("between");
    yield return 2;
    Console.WriteLine("end");
}

var seq = Counting();               // prints nothing at all
foreach (var n in seq)              // start, 1, between, 2, end
    Console.WriteLine(n);

The interleaving is the point. Control ping-pongs between the loop and the iterator body, one item at a time.

There is also yield break, which ends the sequence early, the iterator equivalent of return:

public static IEnumerable<Book> UntilOutOfBudget(IEnumerable<Book> source, int budget)
{
    var spent = 0;
    foreach (var book in source)
    {
        spent += book.Price;
        if (spent > budget) yield break;   // stop the sequence here
        yield return book;
    }
}

What laziness buys you

Three things you cannot get from a List<T>.

You can stream data larger than memory. This reads a million-line file and never holds more than one line:

public static IEnumerable<string> ReadLines(string path)
{
    using var reader = new StreamReader(path);
    string? line;
    while ((line = reader.ReadLine()) != null)
        yield return line;
}

var firstMatch = ReadLines("catalogue.csv").First(l => l.Contains("Orwell"));

First stops asking after the match, so the loop stops reading, and the using disposes the reader on the way out. A 2 GB file, a few kilobytes of memory. Building a List<string> first would need all 2 GB.

You can express infinite sequences. Nonsense for a list, fine for an iterator:

public static IEnumerable<int> Naturals()
{
    var i = 0;
    while (true) yield return i++;
}

var firstTen = Naturals().Take(10).ToList();   // terminates, because Take stops asking

The sequence is infinite; the consumer decides when to stop. This is the whole reason LINQ operators are lazy: the consumer, not the producer, controls how much work happens.

Operators fuse into a single pass. In books.Where(...).Select(...).Take(3), each item flows through the whole chain before the next one starts, and the source is abandoned after three matches. No intermediate lists are allocated at any stage. Chain three eager operations instead and you have built three throwaway collections.

Where laziness bites

Laziness is a genuine trade, not a free upgrade.

Multiple enumeration. This is the bug you will actually hit:

IEnumerable<Book> expensive = books.Where(b => b.Price > 10);

Console.WriteLine(expensive.Count());        // runs the filter
foreach (var b in expensive) { ... }         // runs the filter AGAIN

For an in-memory list that is wasted CPU. For a database query or a file, that is two round trips and possibly two different answers. Compilers and analysers flag this as “possible multiple enumeration of IEnumerable”. The fix is ToList() once, then reuse it.

Exceptions surface late. An argument check inside an iterator method does not run when the method is called, it runs on the first MoveNext(), which may be far away in the stack. That is why validating iterator methods are usually written as a pair: a normal method that validates and then returns the private iterator.

public static IEnumerable<Book> Cheapest(IEnumerable<Book> source, int count)
{
    if (source is null) throw new ArgumentNullException(nameof(source));   // throws now
    return Iterator();

    IEnumerable<Book> Iterator()                                            // throws on MoveNext
    {
        foreach (var b in source.OrderBy(b => b.Price).Take(count))
            yield return b;
    }
}

The source can change underneath you. Modify a List<T> while a foreach over it is in flight and you get InvalidOperationException: Collection was modified. That check exists precisely because the enumerator holds a position that mutation would invalidate.

You cannot use yield return inside a try block that has a catch, and iterator methods cannot have ref or out parameters. Both fall out of the state machine rewrite.

IEnumerable, ICollection, IList: what to return

A quick hierarchy, because the interview version of this question is really about API design:

• IEnumerable<T>: you can iterate it, once, forwards. No Count, no indexer.
• ICollection<T>: adds Count, Add, Remove, Contains.
• IList<T>: adds indexing by position and Insert.
• IReadOnlyList<T>: Count and an indexer, no mutation.

The practical guidance: accept the weakest, return the strongest thing that is honest. Take IEnumerable<T> as a parameter so callers can pass anything. But returning IEnumerable<T> from a repository is a real trap:

public IEnumerable<Book> GetBooks() => _db.Books.Where(b => b.IsActive);   // risky

The caller cannot tell whether that is a finished list or a live database query. If they enumerate it twice, they hit the database twice. If they enumerate it after the DbContext is disposed, they get an exception from deep inside EF Core. Return IReadOnlyList<T> (with the ToList() inside the method) when the work is done, and only return IEnumerable<T> when you deliberately mean “this is a lazy stream, enumerate it once”.

The same idea inside Google, Netflix and the Python world

The iterator protocol is one of the most universally copied ideas in programming, and yield usually keeps its name.

Python. Generators are the same feature with the same keyword, and PEP 255 explicitly cites the same motivation:

def expensive_books(source, threshold):
    for book in source:
        if book.price > threshold:
            yield book

Python’s for loop calls __iter__ then __next__, which are GetEnumerator and MoveNext with different names, and StopIteration is MoveNext returning false.

JavaScript (Meta, every browser). ES6 added both halves at once: the iterator protocol and generator functions.

function* expensiveBooks(source, threshold) {
  for (const book of source) {
    if (book.price > threshold) yield book;
  }
}

for...of calls next() and reads { value, done }. Same cursor, same pause-and-resume.

Kotlin (Google, Android). Sequences are lazy chains built on the same idea, and the contrast with eager list operators is explicit in the API:

val first = books.asSequence().filter { it.price > 10 }.first()

books.filter { } builds a new list; books.asSequence().filter { } streams. Kotlin makes you choose per call what C# chooses per interface.

Java (Netflix). Iterable and Iterator with hasNext()/next() are the same pair, and Java streams are the lazy layer on top. Java has no yield for iterators, which is exactly why writing a custom Java Iterator still looks like the forty-line hand-rolled class above.

And a bonus connection: the pause-and-resume state machine that yield return generates is the same compiler technique used for async/await, which shows up later in this series. Both rewrite a method into an object that remembers where it got to.

Summary

• foreach is sugar for GetEnumerator(), then MoveNext() and Current in a loop, with a Dispose() in a finally.
• IEnumerable<T> is the sequence; IEnumerator<T> is the cursor over it. One sequence can hand out many cursors.
• yield return makes the compiler generate that cursor class for you as a state machine.
• An iterator method runs no code when called. Each yield return hands out one value and freezes the method; the next MoveNext() resumes it.
• This is the mechanism behind LINQ’s deferred execution, and it enables streaming huge sources, infinite sequences, and single-pass operator chains with no intermediate allocations.
• The costs: multiple enumeration repeats the work, exceptions surface on first iteration rather than at the call, and the source must not change mid-iteration.
• Accept IEnumerable<T>; return IReadOnlyList<T> when the work is finished, and IEnumerable<T> only when you truly mean “lazy stream”.

---

Interview questions and answers

1. What is the difference between IEnumerable and IEnumerator?

IEnumerable<T> represents a sequence and exposes one method, GetEnumerator(). IEnumerator<T> is the cursor that actually walks it, with Current, MoveNext() and Reset(). The sequence is the collection; the enumerator is the position in it, which is why several enumerators can iterate one sequence independently.

2. What does foreach compile into?

A call to GetEnumerator(), then a while (enumerator.MoveNext()) loop reading enumerator.Current, wrapped in a try/finally that disposes the enumerator if it implements IDisposable. The type does not strictly have to implement IEnumerable; the compiler pattern-matches on a suitable GetEnumerator method.

3. What does yield return do?

It turns the containing method into an iterator: the compiler rewrites it into a state machine class implementing IEnumerator<T>. At runtime, yield return supplies the next value and suspends the method with all its locals intact, resuming on the following line when the consumer asks for the next item.

4. When does the body of an iterator method execute?

Never at the point of call. Calling it only constructs and returns the state machine. The body starts running on the first MoveNext(), and runs a piece at a time thereafter, which is why argument validation in an iterator method throws late unless you split it into a validating wrapper plus a private iterator.

5. What is the difference between yield return and yield break?

yield return produces a value and suspends. yield break terminates the sequence immediately, making the next MoveNext() return false. It is the iterator equivalent of return in a normal method.

6. How does yield return relate to LINQ’s deferred execution?

It is the implementation. Operators like Where, Select and Take are extension methods whose bodies use yield return, so calling them builds a state machine instead of doing work. The work happens when a terminal operator or a foreach starts pulling items through the chain.

7. What is multiple enumeration and why is it a problem?

Enumerating the same lazy IEnumerable more than once re-executes the entire pipeline. On a list it wastes CPU; on a database query or a file it means repeated I/O and possibly inconsistent results between passes. Materialise with ToList() once when you need the results more than once.

8. Can you use yield return inside a try/catch?

Not inside a try block that has a catch clause. It is allowed inside a try with only a finally, and inside the try of a using, and the finally runs when the enumerator is disposed, including when a consumer stops early or breaks out of the loop.

9. Why is returning IEnumerable from a repository method risky?

The caller cannot see whether they hold results or an unexecuted query. They may enumerate it twice and hit the database twice, or enumerate it after the DbContext has been disposed and get a runtime exception. Returning IReadOnlyList<T> or List<T> states plainly that the work is already done.

10. How can an infinite sequence work without hanging the program?

Because iteration is pull-based: the producer only advances when the consumer calls MoveNext(). An iterator with while (true) yield return i++; produces values on demand, and an operator like Take(10) or First() simply stops asking, at which point the producer stops running.