Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AutoComplete with StackExchange.Redis works fast on Windows server while it works very slow on Linux server #3387

Closed
praveenpnj94 opened this issue Jul 12, 2016 · 4 comments

Comments

@praveenpnj94
Copy link

Hi,
I have written an Autocomplete Console application using Redis in C#. The application adds "Names" as keys and "Code" as values from SQL Database. After that, it searches the matching pattern using SearchKeys and displays the Name matching with the pattern given by the user. It works fine on localhost and windows server but takes a very long time on Linux Server. What could be the possible solution for my problem?

@badboy
Copy link
Contributor

badboy commented Jul 12, 2016

Without any code it could be everything.

@praveenpnj94
Copy link
Author

`using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using LucidFox.CommonLib.Redis;
using System.IO;
using System.Diagnostics;
using System.Data.SqlClient;
using System.Data.Sql;
using System.Data;
namespace Searching
{
class Program
{
static void Main(string[] args)
{

        //Adding the database into Redis using method
        // WriteIntoRedis();
        //Pattern finding

        string[] begState = { "Equity", "F&O", "MF" };
        Console.WriteLine("Please enter the type where you want to search");
        Console.WriteLine("Press 1 for Equity details");
        Console.WriteLine("Press 2 for Future and Option details");
        Console.WriteLine("Press 3 for Mutual Fund details");
        Console.WriteLine("Press 4 for Overall details");
        int type = Convert.ToInt32(Console.ReadLine());
        while (type > 4)
        {
            Console.WriteLine("Please make valid request");
            type = Convert.ToInt32(Console.ReadLine());
        }
        Stopwatch sw = Stopwatch.StartNew();
        TransferAgent(type, begState);
        sw.Stop();
        var timespan = sw.ElapsedMilliseconds;
        Console.WriteLine("Elapsed Time: {0} ms", timespan);

        //Flush Redis
        //Redis.FlushRedisAsync();
    }
    public static void WriteIntoRedis()
    {
        string connectionstring = "Data Source=10.250.26.19;Initial Catalog=LucidFox;Persist Security Info=True;User ID=lucidfox;Password=lucidfox~123";
        using (SqlConnection conn = new SqlConnection(connectionstring))
        {
            conn.Open();
            SqlCommand command1 = new SqlCommand("SELECT CompanyCode,CompanyName FROM vwEquityDetails", conn);
            using (SqlDataReader reader1 = command1.ExecuteReader())
            {
                while (reader1.Read())
                {
                    int index1 = reader1.GetInt32(0);
                    string name1 = reader1.GetString(1);
                    string name1key = "Equity-" + name1;
                    Redis.Add(name1key, index1);

                }
            }

            SqlCommand command2 = new SqlCommand("select [CompanyName]+' ' + Case when OPTType='XX'then 'Future ' when OPTType='CE'then 'Call ' when OPTType='PE'then 'Put '  End +Convert(varchar,ExpDate,106)+' '+Convert(varchar(10),strikeprice) As SearchKey,A.Symbol,Expdate,StrikePrice,OPTType from [dbo].[vwEquityDetails] A inner join vwFutureAndOption B on A.Symbol=B.Symbol --and B.Symbol='ABIRLANUVO' order by [CompanyName],Expdate,OPTType,StrikePrice", conn);
            using (SqlDataReader reader2 = command2.ExecuteReader())
            {
                while (reader2.Read())
                {
                    string a = reader2.GetString(1);
                    var b = reader2.GetDateTime(2);
                    var c = reader2.GetDouble(3);
                    string d = reader2.GetString(4);
                    string code = a + "|" + b.ToString() + "|" + c.ToString() + "|" + d;
                    string name2 = reader2.GetString(0);
                    string name2key = "F&O-" + name2;
                    Redis.Add(name2key, code);
                }
            }
            SqlCommand command3 = new SqlCommand("SELECT [mf_schcode],[SCH_NAME] FROM [dbo].[vwMFDetails]", conn);
            using (SqlDataReader reader3 = command3.ExecuteReader())
            {
                while (reader3.Read())
                {
                    int index3 = reader3.GetInt32(0);
                    string name3 = reader3.GetString(1);
                    string name3key = "MF-" + name3;
                    Redis.Add(name3key, index3);

                }
            }

        }
    }
    public static void TransferAgent(int selection, string[] prefix)
    {
        Console.WriteLine("Please enter the expression: ");
        string ipText = Console.ReadLine();
        int num = 0;
        int sum = 0;
        int othernum = 0;
        if ((prefix.Length - selection) == -1)
        {
            while (selection > 1)
            {
                num = GetResult((selection - 1), prefix, ipText);
                selection--;
                sum = sum + num;
            }

            Console.WriteLine("Total matches found is " + sum);


        }
        else
        {
            othernum = GetResult(selection, prefix, ipText);
            Console.WriteLine("Total matches found is " + othernum);
        }
    }
    public static int GetResult(int p, string[] prefix, string text)
    {
        string beginning = prefix[p - 1];
        string pattern = beginning + "-" + text + "*";
        var result = Redis.GetAllByPattern(pattern).Result;
        int a;
        if (result != null)
        {
            a = result.Count;
            foreach (var item in result)
                Console.WriteLine("Company found is {0} and the code associated is {1} ", item.Key, item.Value);
        }
        else
        {
            a = 0;
            Console.WriteLine("No match found");
        }
        return a;
    }
}

}
`

Redis.cs

`using StackExchange.Redis;
using StackExchange.Redis.Extensions.Core;
using StackExchange.Redis.Extensions.Newtonsoft;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Configuration;
using System.Threading.Tasks;
using System.Collections;
using Newtonsoft.Json.Linq;
using Newtonsoft.Json;

namespace LucidFox.CommonLib.Redis
{
public class Redis
{
private static Lazy connection = null;
private static StackExchangeRedisCacheClient cacheClient = initRedis();

    public static ConnectionMultiplexer Connection
    {
        get
        {
            return connection.Value;
        }
    }

    private static StackExchangeRedisCacheClient initRedis()
    {
        try
        {
            ConfigurationOptions config = new ConfigurationOptions();
            config.EndPoints.Add(ConfigurationManager.AppSettings.Get("RedisHostName"));
            config.AllowAdmin = Convert.ToBoolean(ConfigurationManager.AppSettings.Get("RedisAllowAdmin"));
            config.SyncTimeout = Convert.ToInt32(ConfigurationManager.AppSettings.Get("RedisSyncTimeOut"));
            config.ConnectTimeout = Convert.ToInt32(ConfigurationManager.AppSettings.Get("RedisConnectTimeOut"));
            config.AbortOnConnectFail = Convert.ToBoolean(ConfigurationManager.AppSettings.Get("RedisAbortConnect"));
            // if abortconnect fail is set, then the connect call will not throw exception and redis retries connection in the background thread

            connection = new Lazy<ConnectionMultiplexer>(() =>
            {
                try
                {
                    return ConnectionMultiplexer.Connect(config);
                }
                catch (Exception)
                {
                    //Log
                    return null;
                }
            });
        }
        catch (Exception)
        {
            throw;
        }

        cacheClient = new StackExchangeRedisCacheClient(Connection, new NewtonsoftSerializer());
        return cacheClient;
    }

    ///Do not use Static Constructors. CA1810
    //static RedisController()
    //{
    //    try
    //    {
    //        var conn = ConnectionMultiplexer.Connect("10.250.1.167:6379,allowAdmin=true,syncTimeout=500000,connectTimeout=5000");
    //        var serializer = new NewtonsoftSerializer();
    //        cacheClient = new StackExchangeRedisCacheClient(conn, serializer);
    //    }
    //    catch (Exception ex)
    //    {
    //        throw ex;
    //    }
    //}

    public static bool Add(string key, object value)
    {
        try
        {
            return cacheClient.Add(key, value);
        }
        catch (Exception ex)
        {
            throw ex;
        }
    }

    public static bool Add(string key, object value, double timeOut)
    {
        try
        {
            return cacheClient.Add(key, value, TimeSpan.FromMilliseconds(timeOut));
        }
        catch (Exception ex)
        {
            throw ex;
        }
    }

    public static IList<T> GetList<T>(string key)
    {
        IList<T> list = null;
        try
        {
            Object obj = cacheClient.Get<Object>(key);
            JArray arr = new JArray();
            if (obj != null)
            {
                if (obj.GetType() == arr.GetType())
                {
                    list = JsonConvert.DeserializeObject<List<T>>(JsonConvert.SerializeObject(obj));
                }
                else
                {
                    list = new List<T>();
                    list.Add(JsonConvert.DeserializeObject<T>(JsonConvert.SerializeObject(obj)));
                }
            }
        }
        catch (Exception ex)
        {

            throw ex;
        }
        return list;

    }

    public static T Get<T>(string key)
    {
        try
        {
            return cacheClient.Get<T>(key);
        }
        catch (Exception ex)
        {
            throw ex;
        }
    }

    public static bool Exists(string key)
    {
        try
        {
            if (!string.IsNullOrEmpty(key))
                return cacheClient.Exists(key);
            return false;
        }
        catch (Exception ex)
        {
            throw ex;
        }
    }

    //expiresIn in milli seconds
    // -1 expires it immediately and would be flushed by redis in the next poll cycle
    public static bool SetExpiry(string key, double expiresIn)
    {
        try
        {
            if (Exists(key))
            {
                return cacheClient.Database.KeyExpire(key, TimeSpan.FromMilliseconds(expiresIn));
            }
            return false;
        }
        catch (Exception ex)
        {
            throw ex;
        }
    }


    public static bool Remove(string key)
    {
        try
        {
            if (!string.IsNullOrEmpty(key))
                return cacheClient.Remove(key);
            return false;
        }
        catch (Exception ex)
        {
            throw ex;
        }
    }


    //*foo  - remove all keys ending with foo
    //*foo* - remove all keys containing foo in the key
    //fo*o - matches foo, fooo, foooo ...
    public static async Task<bool> RemoveByPattern(string keyPattern)
    {
        try
        {
            if (!string.IsNullOrEmpty(keyPattern))
            {
                var keys = await cacheClient.SearchKeysAsync(pattern: keyPattern);
                if (keys.Count() > 0)
                    await cacheClient.RemoveAllAsync(keys);
                return true;
            }
            return false;
        }
        catch (Exception ex)
        {
            throw ex;
        }
    }

    //not recomended to be used in Production
    //The client calling this menthod should await for the async operation to complete.
    //boxing unboxing should be handled by the client, since keyPattern can return different types of data - represented by IDict
    public static async Task<IDictionary<string, object>> GetAllByPattern(string keyPattern)
    {
        IDictionary<string, object> retData = null;
        try
        {
            if (!string.IsNullOrEmpty(keyPattern))
            {
                var keys = await cacheClient.SearchKeysAsync(pattern: keyPattern);
                if (keys.Count() > 0)
                    retData = await cacheClient.GetAllAsync<object>(keys);
            }
            return retData;
        }
        catch (Exception ex)
        {
            throw ex;
        }
    }

    [System.Diagnostics.Conditional("DEBUG")]
    public static async void FlushRedisAsync()
    {
        await cacheClient.FlushDbAsync();
    }
}

}`

@badboy
Copy link
Contributor

badboy commented Jul 12, 2016

Keep in mind that this issue tracker should be used for bugs or improvements to the Redis server.
I think your request is better suited for the Reddit or IRC.
Ask on /r/redis, the mailing list or ask in the irc channel #redis on freenode.

@filipecosta90
Copy link
Contributor

@praveenpnj94 closing this one as per the reasons clarified by @badboy above. Please follow up on the client project if this issue is still valid.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants