Write dynamic objects

[benfoster]

Is it possible to write an IEnumerable<dynamic>. The following code throws an exception "No properties are mapped for System.Object":

        var foos = new List<dynamic>();
        dynamic obj = new ExpandoObject();
        obj.Foo = "Bar";
        foos.Add(obj);

        var ms = new MemoryStream();
        using (var sw = new StreamWriter(ms))
        {
            using (var writer = new CsvWriter(sw))
            {
                writer.WriteRecords(foos);
            }
        }

[JoshClose]

I will need to look into this. I thought it wasn't working because ExpandoObject is essential a dictionary, but I tried using an anonymous type too and have the same exception.

I will take a look at this tonight.

[JoshClose]

The issue is that when writing it looks at the type of the generic parameter of the IEnumerable that's passed in and uses that to auto map. Since the type is dynamic, it doesn't come up with anything. I changed it to look at the type of the record for each row instead. This means that each row could have a different type in it, and it would still write. The CSV file would not be proper in this case, but it works with dynamics. The header will be written based on the properties of the first record.

Using ExpandoObject still will not work though. The type inherits IEnumerable, which can't be written. I could possibly check for an IDictionary<string, object> specifically and handle it. I will need to check into how much custom work that would be.

I will put out at least the fix for dynamic tomorrow, and I will update you on the dictionary issue.

[benfoster]

Thanks for the update Josh.

We've built a generic reporting system on top of Dapper which is returning dynamic objects. Specifically Dapper returns IEnumerable<FastExpando> a class that inherits DynamicObject. Structurally this is very similar to ExpandoObject and can be cast to IDictionary<string, object>.

Would it be possible to register a CSV Map for these dynamic objects or does the mapping API require compile time properties?

[JoshClose]

I need to handle the IDictionary<string, object> as a special case. It looks like it'll be a fair amount of work. It's on my list of things to do. :)

[kendaleiv]

Any status or tips for implementing the IDictionary<string, object> special case? Is it a matter of implementing an ITypeConverter -- or do these not support more than a single property?

[JoshClose]

No, it doesn't support more than a single property per field for writing.

The change will require it to detect that it's an IDictionary when mapping and use key/value pairs to create the maps instead of the properties.

[mitchsh]

Have you looked at the possibility of using the GetDynamicMemberNames method from the DynamicObject class? Could this be used to build the map? I'm mainly concerned with writing an object that is dynamically built at runtime.

[JoshClose]

I haven't, but good thinking. I'll keep that in mind when I do this. Thanks.

[JoshClose]

Also, looks like I would have to use IDynamicMetaObjectProvider.GetMetaObject(Expression) instead. DynamicObject has GetDynamicMemberNames but ExpandoObject does not. They both inherit from IDynamicMetaObjectProvider though.

[JoshClose]

Update stackoverflow question when this functionality gets implemented. http://stackoverflow.com/questions/26072099/how-do-you-use-csvhelper-to-write-a-class-derived-from-dynamicobject/26150939#26150939

[damiangreen]

I've managed to write lists of expandos, like so:

 public static IEnumerable<dynamic> ToExpandoList(this List<object[]> results, List<string> columns)
        {
            foreach (var item in results)
            {
                IDictionary<string, object> expando = new ExpandoObject();
                for (var i = 0; i < columns.Count; i++)
                {
                    expando.Add(columns[i], item[i]);
                }
                yield return ((dynamic)expando);
            }
        }

 public static MemoryStream GenerateCsv(IEnumerable records, List<string> columnNames)
        {
            var stream = new MemoryStream();
            var writer = new StreamWriter(stream);
            var csv = new CsvWriter(writer);

            columnNames.ForEach(item => csv.WriteField(item));

            csv.NextRecord();

            foreach (var item in records)
            {
                foreach (var v in (IDictionary<string, object>)item)
                {
                    csv.WriteField(v.Value);
                }
                csv.NextRecord();
            }
            writer.Flush();
            stream.Flush();
            stream.Position = 0;
            return stream;
        }

using (var reader = cmd.ExecuteReader())
                    {
                        var columns = reader.GetColumnNames();
                        var results = DataReaderExtensions.Read(reader).ToList();
                        var list = results.ToExpandoList(columns);
                        return GenerateCsv(list, columns);
                    }

Just in case this helps anyone until this functionality is incorporated

[ravensorb]

Its been a few years :) Any update on this?

[JoshClose]

No. :( I'm making this project more of a priority now though so I can get through some of these major features that have been around for a while. Unfortunately, stuff like this tends to take a lot of hours to do.

[JoshClose]

This is turning out to be a really difficult thing to do. The problem is, you can do anything based on the type, because it doesn't mean anything. You need to have the object itself. This doesn't work with reflection and there aren't any PropertyInfos, so the whole structure of how CsvHelper works doesn't really apply. The code is also very ugly because there needs to be #if !NET_2_0 && !NET_3_5 && !PCL directives all over the place. If it was just ExpandoObject, it wouldn't be so bad, but I need to work with IDynamicMetaObjectProvider instead.

I found a way to pull the data here. http://stackoverflow.com/a/14011692/68499 This still has the issues I stated above.

I'm working through it, but it'll probably take a few days at least. Hopefully I can come up with something that is somewhat clean.

[JoshClose]

I found a way that isn't horrible to look at. This feature will be in version 3.0.0-beta2 on NuGet.

[mayconbeserra]

Hi @JoshClose , is there any documentation for writing the csv based on a list of dynamic objects?

I haven't seen anything in the documentation.

Regards,

[JoshClose]

Not yet, but it should work like any other type of object. Are you having some issues with it?

[CheeseNPort]

@JoshClose , thanks for this, I was also looking at writing ExpandoObjects to Csv's and having installed version 3.0.0 beta 7 it all worked well!! I tried to install the latest which was beta 8 but got a Nuget error saying it wasn't compatible with .Net version 4.5.2.

I would have suggested using the Dynamitey package which is brilliant for getting a list of dynamic properties and their values from a dynamic object but as you've sorted it, I don't really mind how it works! :)

Great library, thanks for this.

[CheeseNPort]

And @mayconbeserra, using version 3.0.0 beta 7, I just called the code as normal i.e.

public byte[] GetCsvFile()
        {
            dynamic Expando1 = new ExpandoObject();
            Expando1.Foo1 = "Bar1";
            Expando1.Foo2 = "Bar2";

            dynamic Expando2 = new ExpandoObject();
            Expando2.Foo1 = "Bar3";
            Expando2.Foo2 = "Bar4";

            List<dynamic> ListToExport = new List<dynamic>()
            {
                Expando1,
                Expando2
            };

            using (MemoryStream Ms = new MemoryStream())
            {
                using (StreamWriter Sw = new StreamWriter(Ms))
                {
                    CsvWriter OutputCsv = new CsvWriter(Sw);
                    OutputCsv.WriteRecords(ListToExport);
                }
                return Ms.ToArray();
            }
        }

[JoshClose]

So, can I ask what your use case for .NET 4.5.2 is? Why are you using that over 4.6?

I switched to VS2017 and removed all project types except netstandard1.3. Here is a chart of what that supports. https://github.com/dotnet/standard/blob/master/docs/versions.md

If it's a common case to need < 4.6 for new development, I'll add back in 4.0 support or something. My thought is new development would be >= 4.6 or core. Legacy development could use CsvHelper 2.x.

[CheeseNPort]

Fortunately I am in a position to upgrade to 4.6 without any problem's so I'll do that. No reason, just because that was the version when the project was created,

[pwen090]

One reason supporting older .NET might be helpful is I believe that Mono currently maps to .NET 4.5 for support. So it might be helpful in being compatible with Mono which helps to be able to use the code across OSX and Linux. I could be missing some Mono/.NET 4.6 compatibility nuance so apologies in advance!

[JoshClose]

Xamarin can use .NET Core. https://blog.xamarin.com/net-standard-library-support-for-xamarin/

Are you talking about something else?

[pwen090]

Mono is a bit different than Xamarin. The default Mono project targets .NET 4.5. I just tested a Mono project with CsvHelper and you can properly add and use CsvHelper 2.16.3. But when you try to add the latest beta it gives an error because the package does not target .NET 4.5 etc... There might be some way to leverage the net standard beta version of CsvHelper within a Mono project but I am not sure. I will let you know if I figure out a way but just wanted to mention it.

[JoshClose]

Based on this https://github.com/dotnet/standard/blob/master/docs/versions.md you should be able to use Mono 4.6 with netstandard1.3, which is what CsvHelper 3.0 is using. Are there any issues using 4.6 over 4.5?

[ghost]

I'm struggling with this a little. It's easy enough to serialize a sequence of ExpandoObjects to CSV. However, I'm concerned about the safety of the code. Is there any guarantee of the ordering of the properties in the ExpandoObject? Given that it implements IDictionary it doesn't seem like there is an ordering guarantee of the properties. If the implementation of ExpandoObject (or it's internal dictionary) changes in the future, how do I ensure that my properties/columns are enumerated in the same order? It seems like that makes this a risk (albeit probably a low one...)

dynamic a = new ExpandoObject();
a.Name = "Foo";
a.Val = 19;
a.Bonus = "My customer is expecting this in the 3rd column";

dynamic b = new ExpandoObject();
b.Name = "Bar";
b.Val = 20;

using (var memStream = new MemoryStream())
{
    using (var sw = new StreamWriter(memStream))
    using (var writer = new CsvWriter(sw))
        writer.WriteRecords(new[] { a, b });
    Console.WriteLine(Encoding.UTF8.GetString(memStream.ToArray()));
}

[JoshClose]

If you need to have it in a specific order and don't want to count on just the header names, you'll need to use an actual class and specify a column index; either by using a mapping file, or attributes.

[sintetico82]

Hi, I am using the 2.16.3
In which version is it possible to use IDictionary<string, object>?

[JoshClose]

Use it how? Does this pertain to dynamic objects?

[mishal153]

Looking for ways to map the dynamic object property to 'CSV header name' of their own. seems related and impossible to do at the moment. This is a promising option at the moment: https://askcodes.net/questions/dynamic-creation-of-columns-using-csvhelper

[JoshClose]

@mishal153 Do you want to use dynamic types or are do you want to create a map at runtime?