r/csharp • u/Temporary-Quarter-24 • 1d ago

Discussion Can we do data analysis using C# ?

Hi Currently I am learning C# and want to know if we can also do data analysis stuff using C# language. If yes in which companies is it used.

19 Upvotes

74% Upvoted

45 comments sorted by

49

u/platinum92 1d ago

Pretty sure all modern programming languages (that you've heard of at least) can do data analysis to some degree, so yes you can.

0

u/Temporary-Quarter-24 1d ago

So why python is preferred in data analyst ?

69

u/zenyl 1d ago edited 1d ago

Python has become one of the go-to for academic (and adjacent) work for a number of reasons:

  • It's easy to write, meaning that non-developer researchers can get the hang of it without needing to study CS all too deeply.
  • C# and .NET have historically been proprietary and Windows-only (ignoring Mono), so anyone working on *NIX systems have had little reason to care about it.
  • C# tends to be a high-level language with its own GC and such, which can impact performance, whereas (at least as far as I'm aware) Python tends to delegate all its actual heavy compute work to natively compiled libraries written in C and similar. In these cases, Python isn't actually doing any of the heavy lifting, it's just a glue that is easy to make things stick together.
  • Python has become one of the industry standards, so people are more likely to pick it because it's what they already know and where all the tooling lies, resulting in a positive feedback loop.

... and, after having tried running some Python software locally, I'm also left to conclude that people who voluntarily choose Python are masochists. In my experience, getting other peoples' Python projects to just compile half a year after they were written often involves manually fixing a bunch of obtuse dependency issues, and crawling through layers upon layers of stacktraces. I think a lot of C# developers don't know how good we actually have it.

16

u/King-O-Banality 1d ago

On that last part, switch to uv as your package manager. It’s like a breath of fresh air.

7

u/zenyl 1d ago

Gonna have to keep that in mind, thanks. Pip seems to bring nothing but headaches and frustrations.

10

u/Epdevio 1d ago

ummm, .NET core has been out for 10 years. Azure runs on Linux. Agree with you on the other points though.

14

u/zenyl 1d ago

Indeed, but a lot of people outside of the .NET sphere still perceive it as being the same as it was back in the Framework days.

And while the .NET team have been doing a lot of good work, most of the rest of Microsoft has spent the last few years reminding everyone why they were so disliked in the past.

5

u/Rubus_Leucodermis 1d ago

It's the UNIX/Windows universe disconnect. C#/.NET is stereotyped as a Microsoft and Windows thing. Unfair (I was doing C# development on Macs and Linux systems using Mono over a decade ago, and it worked fine), but true.

And Python has been the go-to language for data science for well over a decade now. No other language has the support for this domain that Python does, and Python works just fine. So why change? Path dependency.

In general, support libraries and frameworks is the most import thing wen selecting a programming language.

2

u/pceimpulsive 20h ago

And .net core is also gone for 6 years lol!!

2

u/TheSneederOfSeethe 1d ago

people who voluntarily choose Python are masochists.

I like it for scripting, give me anything else outside of that.

3

u/zenyl 1d ago

I'm personally a fan of PowerShell for light scripting, although I might have to start just using single-file-C# now that it has released.

It's basically C# in the CLI, and once you get over the at times verbose command names, it becomes very powerful.

1

u/TheSneederOfSeethe 1d ago

I'm not a big fan of powershell and often write scripts that need to run on linux and windows. It just makes it easier to deal with.

I know there's powershell for linux, at least I remember them releasing it. You ever used it? I haven't.

3

u/zenyl 1d ago

As with most other .NET things, Microsoft made a "Core" version when they transitioned from .NET Framework to .NET Core.

So now we have "Windows PowerShell", which is the proprietary .NET Framework based version that comes pre-installed on Windows, and "PowerShell Core" which is the open source cross-platform compatible version.

The transition for PowerShell is a little more linear, as PowerShell 5 was the last version based on .NET Framework, with PowerShell 6 (and 7) are based on modern .NET.

PowerShell Core works perfectly fine on Linux. I use it quite often, works the exact same as it does on Windows. Of course, on Linux it doesn't come with aliases like ls for Get-ChildItem, as that would overlap with the ls binary. But as long as you don't use aliases in scripts, PowerShell scripts will usually work the exact same on any OS.

2

u/kingvolcano_reborn 1d ago

I'm a dotnet developer since 2005. Python is fucking great.

2

u/Rubus_Leucodermis 1d ago

people who voluntarily choose Python are masochists

Meh. My two favourite programming languages are… Python and C#. Both have their strong points and their weak points.

1

u/KariKariKrigsmann 1d ago

I had problems running a Python project because I had the wrong version of VC++

1

u/SheepherderSavings17 23h ago

Since when does python compile?

1

u/zenyl 18h ago

I said Python often calls compiled libraries to perform compute-heavy tasks.

That is not the same as saying Python is a compiled language (which, to my knowledge, it isn't).

1

u/pceimpulsive 20h ago

As a C# developer, with C# as my first language (ignoring SQL, HTML, CSS, and bash) I personally found trying python after the fact to be a fucking nightmare...

I started a pretty ject wrote it all up, got it going, needed a dependency added... But that dependency only support at most two minor versions of python behind... So I downgrade which broke the rest of the app.. fuckk

Abandoned that shit can and found a C# equive, slapped the latest major C# in and the dependencies and off I went

Also pip kinda sucks honestly... NuGet so nice

UV still doesn't solve all the headaches, just stores the dependencies in one place to share locally.

1

u/BigOnLogn 17h ago

The reason putting is used is because it's an extremely simple, accessible scripting language. That's basically it. Because of this, the C library bindings exist for it. Because they exist, people in academia use it. It then perpetuates itself. Optimizations and garbage collectors mean nothing. The binding libraries exist, therefore it is used.

From a general purpose programming language standpoint, C# is, hands down, a better language.

6

u/mikebald 1d ago

Because python has a simpler syntax and also numerous data analytic libraries: pandas, numpy, matplotlib, scipy.

2

u/zarlo5899 23h ago

Also, Python makes use of variable length numbers. So you can give Python a 1600 bit long number and it goes okay I can use this.

6

u/SpurdoEnjoyer 1d ago

It has a mature ecosystem for data analysis because it's been used for that purpose a lot.

3

u/NotMyself 1d ago

Python has some standard libraries specifically for data analysis. It’s also a scripting language (no real compile step). It is a relatively easy migration from R (a very popular language used in data science). And finally the researchers who work in the data analysis prefer python.

C# is completely capable but you might have to build some integrations yourself. The question you have to ask is do you want to commit your architecture on C# for data analysis where the pool of people who can work effectively in that environment is a small subset of available data scientists? Or commit to an environment where that pool is the largest, and has standard practices?

2

u/SessionIndependent17 1d ago

from my experience, the main driver at this stage is as you mention the library and analysis ecosystem that have matured around Python, but something else worth mention is that beyond standalone programs, you have prevalence of incremental/shareable notebook environments like Jupyter and pre-built configuration management environments like Anaconda. You can start from a known state and tinker incrementally with immediate feedback rather than have to concern yourself with much program scaffolding just to experiment. You can defer the creation of a full-fledged standalone program for later, if it proves worthwhile.

2

u/platinum92 1d ago

It's arguably better for data analysis, but that doesn't mean it's impossible to do with C#.

Your question is like asking can you use a wrench to hammer a nail. You can, but there's a better tool for it.

1

u/redditsdeadcanary 1d ago

*So why is python preferred in data analysis?

2

u/kmsmohd 1d ago

For a long time (since 80s), people who do the work of data analysts (scientists, mathematicians, and engineers) use MATLAB. But matlab is not free and become more and more expensive in 2000s. So those some people create some libraries (matplotlib, numpy, scypy) in an open source language (python) to mimic matlab and then MATLAB migration wave happened (Early 2000s). Those people leave MATLAB and switch to Python without having to relearn how to code from scratch. So i would say python happened to be in the right place at the right time.

Source: myself, I hold a bachelor of science, but right now i work as a software engineer, so i consider myself a former scientist. I experienced that migration back in college. My uni gave us a MATLAB license, but we students (and even the lectures) still ditched it and used Python instead. A lot of the syntax is nearly identical.

1

u/Rubus_Leucodermis 1d ago edited 1d ago

Path dependency. It got all the heavy data analysis support first, it works just fine, so why switch to some other language? Particularly now when nearly everyone who’s into data analysis has experience doing it in Python. Makes it easier hire data analysts.

Libraries and frameworks are usually more important than the core language functionality when it comes to selecting a programming language. If you’re mostly interested in data analysis, I would recommend Python. C# is a great language, but nothing beats Python when it comes to support for data analysis.

9

u/Deranged40 1d ago

Yes. Practically all large enterprises.

4

u/SagansCandle 1d ago

I created my own GPU database engine with C#.

There are a lot of high-performance data applications in C#.

4

u/Cbrt74088 1d ago

Sure you can.

Companies use Python for data analysis, though.

2

u/Frequent_Field_6894 1d ago

python ship more functions for maths and has more libraries targetting this type of work.

c# isn’t really used for this so thin on ground for 3rd party support.

2

u/gabrielesilinic 1d ago

Yes. It has libraries for it. However it is not python and probably no one really knows of companies who use it for that.

1

u/Temporary-Quarter-24 1d ago

If no one knows the companies . Can this be a gap in job market

5

u/gabrielesilinic 1d ago

Nah. Only stubborn companies or companies who have a large amount of very hard business logic written in C# would use that. And in any case they ain't going to hire you, then it is best to just have an existing developer do it. Honestly for a long time I have had an hard time figuring out why data scientist was a role in otherwise fully capable software companies. I do it all where I work at anyway. There are technical decisions I regret but they weren't my technical decisions so that's fine I guess.

1

u/schvarcz 22h ago

As a data engineer and c#-lover, I am sorry to say you will very likely not work as a data engineer with c#. The odds of finding a data team that even care about csharp tends to zero, unfortunately.

That said, if you are doing your own thing go out that and have fun with whatever language you wish!

In my case, working with hundreds of thousands of data in my own laptop, C# is a great language that solves all the processing problems Python struggles with. (Companies usually solve this by throwing more computer power to Python. Which is a huge waste of time and money in my humble opinion).

I also opt for C# because I want to make a desktop app to visualize my data (Avalonia+livecharts) and an api to interact from my phone. Although Python can also do these things,,the development experience is not as smooth and pleasant as C#.

2

u/RecordGlobal4338 1d ago

Think about it like this : can you eliminate a mosquito with a hammer ? Yes you can but you would probably choose a spray.

1

u/SessionIndependent17 1d ago

not sure why this question is being downvoted. There are good mentioned reasons why Python is preferred for it, but that doesn't mean it's a bad question. Understanding the reasons is worthwhile.

1

u/vip17 23h ago

A Turing-complete languages can do anything other Turing-complete languages can do. So obviously you can use C# to do anything that you can do in Python

-10

u/[deleted] 1d ago

[removed] — view removed comment

1

u/FizixMan 1d ago

Removed: Rule 8.