Initial DDEX support : new pr

[kenjiuno]

A new pull request.
Initial DDEX support code is inherited from #67 started by wwindcloud.
Note: I have manually rebased previous code. So, it may contain some problems...

DDEX tips:

• DDEX support will work on VS2010/VS2012/VS2013 Pro (VS2013 needs Visual Studio 2013 Update 2 or later) (Edited, 3 Jun)
• Specify all of Host, Username, Password and Database properties in DDEX connection dialog.
• Capture your ConnectionString for later use. For example,

PORT=5432;TIMEOUT=15;POOLING=True;MINPOOLSIZE=1;MAXPOOLSIZE=20;COMMANDTIMEOUT=20;COMPATIBLE=2.1.0.0;HOST=127.0.0.1;USER ID=npgsql_tests;PASSWORD=npgsql_tests;DATABASE=npgsql_tests

Built tips:

• Open Npgsql2012.sln.
• Configuration: Debug-net45, Release-net45 or such.
• Built the entire solution.
• Install the generated vsix. For example, NpgsqlDdexProvider\bin\Debug-net45\NpgsqlDdexProvider.vsix
• gacutil /i Npgsql.dll

(Edit 22 Jun, 2014)
You'll need gacutil for .NET runtime 4.0.
It will be found from one of following folders:

• C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin\NETFX 4.0 Tools\gacutil.exe
• C:\Program Files (x86)\Microsoft SDKs\Windows\v8.0A\bin\NETFX 4.0 Tools\gacutil.exe
• C:\Program Files (x86)\Microsoft SDKs\Windows\v8.1A\bin\NETFX 4.5.1 Tools\gacutil.exe

[samy2]

Hi @kenjiuno,

I tested this pull request over the ngsql master branch. This works great !

My configuration :
Visual Studio 2003 Update 2 RC
Visual Studio SDK
Postgresql 8.1.10 under Linux

To make it work with my version of postgresql, i had to patch GetConstraints and GetConstraintColumns in npgsqlSchema.cs (use of "as" SQL keyword).
(see my pull request : kenjiuno#1).

Then, I installed NpgsqlDdexProvider.vsix and put npqsql.dll, Npgsql.EntityFramework.dll and Npgsql.EntityFrameworkLegacy.dll into the GAC.
After that, all worked perfectly : I can create datasources, view data, and I was also able to use the Entity Framework 6.1.0 Tools for Visual Studio to reverse engineer my database.

I hope this pull request will soon be integrated into the official npsql master branch.

[franciscojunior]

Excellent work, @kenjiuno !
I'll have a look at this.

@samy2 , thanks for giving it a try and let us know it worked ok. I'll work to get it integrated into master.

[franciscojunior]

Hi, @kenjiuno !

Do you think it would be possible to get it working with visual studio 2010?

This is the only Pro version I have. Extensions don't work with express editions :(

I'll check the 2013 evaluation option.

Thanks in advance.

[kenjiuno]

Hi @franciscojunior

I think it will work on earlier versions like VS2010.
I'll find a way to get it work.

[franciscojunior]

Hi, @kenjiuno !

Thanks for having a look at this!

Do you think it would be possible to get it working with visual studio 2010?

Actually, I meant get it compilable in vs.net 2010 :)

[franciscojunior]

Hi, @kenjiuno !

I think this pr is almost ready to be merged. We even have a test done by @samy2 ! :)

I have an observation.

In your changes, I noticed that in the 2002 solution file, there is a signature change:

+# Visual Studio 2013
 +VisualStudioVersion = 12.0.30324.0
 +MinimumVisualStudioVersion = 10.0.40219.1

I think visual studio 2013 changed the vs 2012 project signature.

Can you confirm that? If so, would it be possible to change the vs.net 2013 solution file instead of the 2012?

Thanks in advance.

[kenjiuno]

Hi.

Can you confirm that? If so, would it be possible to change the vs.net 2013 solution file instead of the 2012?

Yes, you are right.
I have used VS2013 pro eval on vs 2012 project. It has changed the signature, as you mentioned.

I will fix problems on VS2012/VS2013 versions, when my MSDN license is activated that I have purchased last week. It will take a week...

also working on VS2010 version...

[kenjiuno]

Hi.

I have updated Vs2010/Vs2012/Vs2013 solutions for NpgsqlDdexProvider.

[kenjiuno]

NpgsqlDdexProvider build tips

VS2010 users

You'll need VS2010 Professional or greater.

SP0 users:

• Install: Visual Studio 2010 SDK
  http://www.microsoft.com/en-us/download/details.aspx?id=2680

SP1 users:

• Install: Microsoft Visual Studio 2010 Service Pack 1 (Installer)
  http://www.microsoft.com/en-us/download/details.aspx?id=23691
• Install: Visual Studio 2010 SP1 SDK
  http://www.microsoft.com/en-us/download/details.aspx?id=21835

If you need newer NpgsqlDdexProvider2010.pkgdef, create your own manually.
pkgdef is a kind of registry file for our DDEX registration.
Note: It is needed only if you change contents of NpgsqlDataProviderRegistration class.

Command example:

H:\Dev\Npgsql\NpgsqlDdexProvider>"H:\Program Files (x86)\Microsoft Visual Studio 2010 SDK SP1\VisualStudioIntegration\Tools\Bin\CreatePkgDef.exe" /out=NpgsqlDdexProvider2010.pkgdef /codebase bin\Release-net40\NpgsqlDdexProvider.dll

Output:

Visual Studio (R) PkgDef Creation Utility.
Copyright (c) Microsoft Corporation. All rights reserved.

CreatePkgDef : warning : The Assembly specified at 'bin\Release-net40\NpgsqlDdexProvider.dll' cannot be loaded because an alternate copy with the same identity
exists in the Assembly probing path at 'H:\Dev\Npgsql\NpgsqlDdexProvider\bin\Release-net40\NpgsqlDdexProvider.dll'. The Assembly at 'H:\Dev\Npgsql\NpgsqlDdexPro
vider\bin\Release-net40\NpgsqlDdexProvider.dll' will be loaded instead.
Assembly: NpgsqlDdexProvider 1.0.0.0
Output file: NpgsqlDdexProvider2010.pkgdef

インストールされている製品:   NpgsqlDdexProviderPackage、Version 1.0
パッケージ:          NpgsqlDdexProviderPackage {958b9481-2712-4670-9a62-8fe65e5beea7}
サービス:          PostgreSQL Provider Object Factory

SUCCEEDED:        NpgsqlDdexProvider

Check: How to create a pkgdef file for your Visual Studio Package http://blogs.msdn.com/b/dsvst/archive/2010/03/08/how-to-create-a-pkgdef-file-for-your-visual-studio-package.aspx

VS2012 users

You'll need VS2012 Professional or greater.

• Install: Microsoft Visual Studio 2012 SDK
  http://www.microsoft.com/en-us/download/details.aspx?id=30668
• Install: Visual Studio 2012 Update 4
  http://www.microsoft.com/en-us/download/details.aspx?id=39305

VS2013 users

You'll need VS2013 Professional or greater.

• Install: Microsoft Visual Studio 2013 SDK
  http://www.microsoft.com/en-us/download/details.aspx?id=40758
• Install: Visual Studio 2013 Update 2 RC or later
  http://www.microsoft.com/en-us/download/details.aspx?id=42307

[franciscojunior]

That's excellent, @kenjiuno !

I'll copy your instructions to a new wiki page NpgsqlDdexProvider!

I'll make some tests with visual studio 2010 and will give you my feedback!

Thank you very much for your support and hard work. You rock!

[franciscojunior]

Wiki page done:
https://github.com/npgsql/Npgsql/wiki/Visual-Studio-Design-Time-Support---DDEX-Provider

[kenjiuno]

@franciscojunior Thanks. It'll be good information source!

[franciscojunior]

@franciscojunior Thanks. It'll be good information source!

You are welcome!

And thank you very much for all your help and support.

I could get it to work in my visual studio 2010. But I only did some simple tests.

I'll try some other tests and will let you know if I find any problems.

I think we are ready to merge it! :)

[franciscojunior]

@kenjiuno , did you check the change proposed by @samy2 ?

[franciscojunior]

Thanks for merging!

You are welcome! And thank you for your hard work!

I'll check if binaries can be built on master branch with VS2012/VS2013. I'll post vsix installer screen shots, too.

Great!
Please, let us know how it goes with VS2012 too.

I can confirm that the binaries can be built on master branch on our build server: https://build.npgsql.org/viewLog.html?buildId=1354&tab=artifacts&buildTypeId=npgsql_all
We even have the vsix package build for vs2013. ;) I'll need to check with @roji how we can setup a build of the 2010 vsix.

[franciscojunior]

I also can confirm the packages are restored correctly when running on Xamarin Studio under Mono on OS X :)

[kenjiuno]

Hi.

Here are some screenshots:

NpgsqlDdexProvider_VSIX_Installer

Installation_Complete

This_extension_is_already_installed_to_all_applicable_products

NpgsqlDdexProvider_Tools_Extensions_and_Updates

Connect_to_Database

Add_Connection

Server_Explorer

Sorry, I have noticed that table names having upper-case won't be previewed. I'll work on it.

[roji]

@kenjiuno, thanks for the screenshots! @franciscojunior, it's probably a good idea to have a small guide with these in the wiki?

@franciscojunior, regarding your git question on removing and .gitignore. .gitignore only has an effect on untracked files. So it's perfectly OK to remove files from git and add them to .gitignore in the same commit.

[franciscojunior]

@kenjiuno, thanks for the screenshots! @franciscojunior, it's probably a good idea to have a small guide with these in the wiki?

Agreed! I'll take care of that. We even have a wiki page about de where I'll add those images and more documentation.

@franciscojunior, regarding your git question on removing and .gitignore. .gitignore only has an effect on untracked files. So it's perfectly OK to remove files from git and add them to .gitignore in the same commit.

Ahhhh , great! I didn't know that. I only had a feeling git would do the right thing though. :) I already merged the gitignore change too.

[kenjiuno]

Please visit Wiki for the latest info →→→ https://github.com/npgsql/npgsql/wiki/Visual-Studio-Design-Time-Support---DDEX-Provider

VS2013Pro+NpgsqlDdexProvider+EF6 how to

Prerequisites:

• Visual Studio 2013 Professional or greater edition
• Microsoft Visual Studio 2013 Update 2 installed. http://www.microsoft.com/en-us/download/details.aspx?id=42666
• PostgreSQL server. Tested on PostgreSQL 9.3.4 (win-x64)

Build your Npgsql

1. Get Npgsql source code. Use git or https://github.com/npgsql/Npgsql/archive/master.zip (Tested with https://github.com/npgsql/Npgsql/archive/841f8b6f157f3209ce8aeb565c75d01de08cb30c.zip).
2. Open [Npgsql2013.sln].
3. Select [Release-net45] at [BUILD]→[Configuration Manager...]
4. Build Solution.

You will obtain 3 important files:

• Npgsql\bin\Release-net45\Npgsql.dll
• Npgsql.EntityFramework\bin\Release-net45\EntityFramework.dll
• NpgsqlDdexProvider\bin\Release-net45\NpgsqlDdexProvider.vsix

Install assemblies into GAC

1. Open [Developer Command Prompt for VS2013] from start menu, [Visual Studio 2013]→[Visual Studio Tools]. Use Run as Administrator!
2. Use gacutil to register [Npgsql.dll] from [Npgsql\bin\Release-net45] folder.
3. Use gacutil to register [Mono.Security.dll] from [lib\Mono.Security\4.0] folder.

How to check installed assembly: gacutil /l Npgsql

C:\Windows\system32>gacutil /l Npgsql
Microsoft (R) .NET Global Assembly Cache Utility.  Version 4.0.30319.33440
Copyright (c) Microsoft Corporation.  All rights reserved.

The Global Assembly Cache contains the following assemblies:
  Npgsql, Version=2.2.0.0, Culture=neutral, PublicKeyToken=5d8b90d52f46fda7, processorArchitecture=MSIL

Number of items = 1

Also: gacutil /l Mono.Security

C:\Windows\system32>gacutil /l Mono.Security
Microsoft (R) .NET Global Assembly Cache Utility.  Version 4.0.30319.33440
Copyright (c) Microsoft Corporation.  All rights reserved.

The Global Assembly Cache contains the following assemblies:
  Mono.Security, Version=4.0.0.0, Culture=neutral, PublicKeyToken=0738eb9f132ed756, processorArchitecture=MSIL

Number of items = 1

Install NpgsqlDdexProvider.vsix

1. Run [NpgsqlDdexProvider.vsix] from [NpgsqlDdexProvider\bin\Release-net45] folder.

Edit machine.config files

1. Add the add-element into DbProviderFactories-element.

<system.data>
  <DbProviderFactories>
    ...
    <add name="Npgsql Data Provider" 
         invariant="Npgsql" 
         description=".Net Data Provider for PostgreSQL" 
         type="Npgsql.NpgsqlFactory, Npgsql, Version=2.2.0.0, Culture=neutral, PublicKeyToken=5d8b90d52f46fda7" 
         support="FF" />
  </DbProviderFactories>
</system.data>

machine.config will be found in following places:

• C:\Windows\Microsoft.NET\Framework\v4.0.30319\Config\machine.config
• C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config\machine.config

Note: Don't forget to launch text editor with Run as Administrator.
Note: Framework64 will be found only for 64-bit Windows.

Prepare new project for testing

1. Launch Visual Studio 2013.
2. [FILE]→[New]→[Project...]
3. [Console Application]
4. Name is [testef] for example.

Install EntityFramework 6

1. Right click project [testef]
2. [Managet NuGet Packages...]
3. Type "EntityFramework" at [Search Online (Ctrl+E)]
4. Install "EntityFramework". Version is 6.1.1 for now.

Add Npgsql.dll and Npgsql.EntityFramework.dll manually

1. Right click project [testef]
2. [Add]→[Existing Item...]
3. Add 2 files you have built:

• Npgsql\bin\Release-net45\Npgsql.dll
• Npgsql.EntityFramework\bin\Release-net45\EntityFramework.dll

Note: Select [All Files (.)] to show your DLL files.

Add references to Npgsql and Npgsql.EntityFramework

1. Right click [References]
2. [Add Reference...]
3. Click [Browse...]
4. Select 2 DLL in your project folder to add them:

Npgsql.dll
Npgsql.EntityFramework.dll

Add EF6 provider into App.config

1. Open [App.config]
2. Add provider-element into providers-element.

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  ...
  <entityFramework>
    ...
    <providers>
      <provider invariantName="Npgsql" type="Npgsql.NpgsqlServices, Npgsql.EntityFramework" />
      ...
    </providers>
  </entityFramework>
  ...
</configuration>

Build once

1. Build your project.

New ADO.NET Entity Data Model

1. Right click project [testef]
2. [Add]→[New Item...]
3. [ADO.NET Entity Data Model]
4. Name is [Model1] for example.
5. Click [Add]
6. [EF Designer from database] at Choose Model Contents.
   
7. [New Connection] at Choose Your Data Connection.
   
8. [PostgreSQL Database] at Change Data Source.
   
9. Fill properties in Connection Properties. It is easy to fill everything by setting [ConnectionString].
   

My sample ConnectionString:

PORT=5432;TIMEOUT=15;POOLING=True;MINPOOLSIZE=1;MAXPOOLSIZE=20;COMMANDTIMEOUT=20;COMPATIBLE=2.1.3.0;HOST=g45;DATABASE=npgsql_tests_ef;PASSWORD=npgsql_tests;USER ID=npgsql_tests;PRELOADREADER=True

Note: Don't forget to set PreloadReader to true.

1. Select [Yes, include the sensitive data in the connection string.] in this case for easy setup.
   
2. Select tables which you want, at Choose Your Database Objects and Settings.
   

Note: remember the text npgsql_tests_efModel at [Model Namespace].

1. Click OK for Security Warning. T4 Templates generator warns you as it contains just runnable C# code.
   
2. You will get a generated model.
   

Edit your program.cs

Just my sample code for npgsql_tests_ef database.

About the name of "npgsql_tests_efEntities" class, check your [Model Namespace] entered above. Replace "Model" with "Entities", like it is "npgsql_tests_efModel".

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace testef
{
    class Program
    {
        static void Main(string[] args)
        {
            using (npgsql_tests_efEntities Context = new npgsql_tests_efEntities())
            {
                foreach (Blogs blog in Context.Blogs)
                {
                    Console.WriteLine(blog.BlogId + " " + blog.Name);
                }
            }
        }
    }
}

Sample output:

[franciscojunior]

Excellent, @kenjiuno !!

I just added your tutorial to our wiki page: https://github.com/npgsql/Npgsql/wiki/Visual-Studio-Design-Time-Support---DDEX-Provider#vs2013pronpgsqlddexprovideref6-how-to

[kenjiuno]

Hi @franciscojunior
It is very helpful. Thank you!

[franciscojunior]

Hi, @kenjiuno !

I thank you for all your hard work!
Domo arigato!

[jubilee-scott]

I am sure this is probably common knowledge, but since I tripped over this for a day I wanted to save anyone else the grief. When performing the above steps using VS 2012, you will have to download and install Entity Framework 6 Tools for Visual Studio 2012 (http://www.microsoft.com/en-us/download/details.aspx?id=40762). Otherwise the "PostgreSQL Database" option will not show up in the "Change Data Source" dialog even though you can connect to the PostgreSQL server using Tools - Connect To Database menu command.

[kenjiuno]

Hi, @jubilee-scott

Thanks for the information. I'll add your information as a trouble shooting note!

And there is another note from user's report that rebuilding project helped to resolve the problem.
https://groups.google.com/d/msg/npgsql-help/rulDcydUAhY/qlUpTCDMKaYJ

Thanks!

[abelondev]

Can you just provide workable sample with EF6 and Npgsql. And please test on Mono as well. I would really helpful for everybody. I tried all advices possible many hours and nothing helps.

[kenjiuno]

Hi @abelondev

Are you targeting .NET development on mono environment?

Can you just provide workable sample with EF6 and Npgsql. And please test on Mono as well. I would really helpful for everybody. I tried all advices possible many hours and nothing helps.

Ok, I want to know development situation like: IDE (SharpDevelop?, command line xbuild?), OS (Windows or Linux or etc)
I'll find the corresponding info, or I might write one for you...

DDEX support is intended to be used with Microsoft Visual Studio 2012/2013/2015 Pro, with Microsoft .NET Framework 4.5 and later.

[dropyghost]

I wondering what is the right sequence of installing new version of Npgsql.

I had the 2,2 and when try to install new 3.0.5 things stop working. I cant create new project and older projects stop working, I guess something was overwritten?

Im using Visual Studio 2013.

I start here http://www.npgsql.org/install.html

Downloading the package from NuGet.

• Install-Package Npgsql -Version 3.0.5
• Install-Package EntityFramework6.Npgsql

Is there a simple way to test if this steps are already working?

Now I want setup the Visual Destign-Time. The main page say try the "experiental installer" (check the typo) but the link is the same page so dont know where that installer is.

When go to "manual installation" but this looks like an older version not sure if work with this new version.

The first instructions say install in the GAC, when the previuos page say you only install Npgsql in the GAC in some cases so is confusing.

Is using the Visual Design one of those cases where you need install in the GAC?

So i go to https://github.com/npgsql/npgsql/releases
and download Npgsql-3.0.5.msi

After install I expect would be updated the machine_config but still the same.

C:\Windows\Microsoft.NET\Framework\v4.0.30319\Config

This is where Im now.

[kenjiuno]

Hi @dropyghost

Sorry for inconvenience, I will rewrite about Visual Studio Design-Time support.

I had the 2,2 and when try to install new 3.0.5 things stop working

It needs Setup_NpgsqlDdexProvider.exe for Npgsql 3.0.5. Please be patient while it is preparing, it takes some days.

I had the 2,2 and when try to install new 3.0.5 things stop working. I cant create new project and older projects stop working, I guess something was overwritten?

Sorry for the problem... can you launch Visual Studio?
If you can, please try open menu [Tools]→[Extensions and Updates...], and then you can manually disable or uninstall NpgsqlDdexProvider.
You can see if it recovers the problem on creating new project.

Is using the Visual Design one of those cases where you need install in the GAC?

NpgsqlDdexProvider for Npgsql 3.x won't use GAC for now. But you can leave Npgsql GAC installed.

NpgsqlDdexProvider for Npgsql 2.2 relies on GAC and machine.config modification.
NpgsqlDdexProvider for Npgsql 3.x will modify Visual Studio's host devenv.exe.config and, supply internal Npgsql.dll embedded to NpgsqlDdexProvider.

[dropyghost]

Thanks @kenjiuno

In december I try 3.0.4 and got similar result, when saw didnt update the GAC I stop trying and because holiday were already near didnt worry to much in ask what was wrong.

I see 3.0.4 already have a Setup_NpgsqlDdexProvider.exe so what steps should I follow to install it?

When I try I follow the step in december I install something but couldnt find those steps now.

Right now I have NpgsqlDdexProvider. version 3.0.3 installed 01/2015

[kenjiuno]

Hi @dropyghost

I have uploaded NpgsqlDdexProvider 3.0.5 at https://github.com/npgsql/npgsql/releases.
Please install it. Older version will be overwritten.

Installing Npgsql 3.0.5 into your project (by NuGet) requires NpgsqlDdexProvider 3.0.5 in order to activate Visual Studio Design Time support.

I see 3.0.4 already have a Setup_NpgsqlDdexProvider.exe so what steps should I follow to install it?

When I try I follow the step in december I install something but couldnt find those steps now.

Yes, the instructions are written across some topics like #718 (comment), sorry.

I have just updated official doc. And it will be reliable source against recent NpgsqlDdexProvider releases. http://www.npgsql.org/doc/ddex

[dropyghost]

I got until the save connection step.

got this message.

My versions looks ok

Already rebuild solution.

Update app.config

Dont understand the part of "remove invariant" after add "invariant"

I try creating a new project and reinstall the Entity Framework 6 from NuGet, but now say is using an older version

[kenjiuno]

Hi. @dropyghost

Your setup and configuration seem to be ok.

Please check the App.config, <configSections> is located at first like this?

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <configSections>
    <!-- For more information on Entity Framework configuration, visit http://go.microsoft.com/fwlink/?LinkID=237468 -->
    <section name="entityFramework" type="System.Data.Entity.Internal.ConfigFile.EntityFrameworkSection, EntityFramework, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" requirePermission="false" />
  </configSections>
  <startup>
    <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />
  </startup>

If <startup> is located at first, please move it.

And, NpgsqlDdexProvider v3.0.5 equips light weight diagnostics feature.

Please try [Check Npgsql project installation] menu item displayed on project's right click.
http://www.npgsql.org/doc/ddex#how-to-check-if-npgsql-ddex-is-working-correctly-npgsql-30x

[dropyghost]

This looks like a bug.

When you try to create a connection and the user doesnt have password you get this error.

Also try adding

;Password=;

In the ConnectionString and didnt work neither

So I just create a new user with password and work ok.

[dropyghost]

@kenjiuno The link is down

http://www.npgsql.org/doc/ddex.html

[kenjiuno]

@dropyghost Hi, please try this link http://www.npgsql.org/doc/ddex

[dropyghost]

@kenjiuno Documentation help link is down again :(

[kenjiuno]

@dropyghost sorry, new link is here http://www.npgsql.org/doc/3.0/ddex.html

[therealmitchconnors]

@kenjiuno it appears that the http://www.npgsql.org/doc/3.0/ddex.html link is down, and the http://www.npgsql.org/doc/ddex link is out of date (it still refers to a setup.exe file instead of vsix).

[kenjiuno]

Hi.

This link is correct! But it is old one targeting NpgsqlDdexProvider 3.0 and below.
http://www.npgsql.org/doc/ddex

I have new NpgsqlDdexProvider 3.1 tutorial expanded on this issue:
#1299
I will migrate to official doc in some days.

[therealmitchconnors]

Thanks, that did the trick. Not having it in the docs is not great, but maybe you could include a link to the updated instructions in the release notes?